Domain-Specific AI Should Focus on Workflows Rather Than Modeling
Having worked on AI for health in the past few years, contributing to Med-Gemini and AMIE, I have witnessed a quite significant shift in the type of research that is required. Typically, domain-specific AI research such as AI for health needed a few years to adopt state-of-the-art models. Of course there are some notable exceptions where adoption happened in the reverse direction, such as U-Net. But usually, modeling research resulting in new architectures or methods was primarily done on general computer vision or natural language processing tasks before slowly getting adopted in health or other domains.
Of course, there was still a fair amount of modeling work being done in health. In computer vision, for example, conferences like MICCAI commonly showcased new or adapted modeling approaches intended specifically for medical imaging. In summary, this means that general models were developed first and then slowly adopted, adapted, and innovated upon for health in domain-specific conferences and journals. This was necessary primarily because general model innovations did usually not perfectly fit their tasks in health. However, by the nature of having to innovate, there was obviously also an element of researchers wanting to work on exciting new models.
Over the past few years, I have seen this shift radically and I believe not all researchers are appropriately prepared or adapting to this shift. More and more, I find that modeling is not necessary anymore when applying general foundation models like multi-modal LLMs in health. Med-Gemini, for example, still required custom encoders to properly reason about multimodal, medical artifacts. MedPrompt investigated test-time scaling for medical problems. Early work on patient-facing AI agents, including AMIE, started with custom post-training to improve conversational quality and medical knowledge. However, more recently, this shifted to using vanilla Gemini with sophisticated multi-agent architectures. In summary, there was always some modeling work required to make models work in the medical domain. However, the modeling work that is actually required to do so is decreasing extremely quickly. We went all the way from custom training to purely focusing on agentic architectures. Now, I am seeing even these complex architectures getting easier and easier to build as general agentic capabilities of the base models improve.
Overall, I feel that exponentially improving general capabilities may make significant domain-specific modeling work obsolete. And I see many researchers underestimating the pace of this development. Yes, there are still areas and problems where general models underperform in health. Real-time audio video interaction may be one of these areas. However, given my experience over the past few years, I expect this to improve radically with the next generation of models. This is because general capabilities will continue to improve rapidly based on current investment. And domain-specific modeling work will not be able to keep up with this speed.
So, what does this mean for domain-specific AI research then?
I believe this is actually incredibly exciting, especially for health. We can actually start focusing on the concrete problems and workflows we want to improve, rather than having to invest in modeling and adapting models to our use-cases. This type of work is often closer to work on benchmarking, safety, human-AI collaboration, human computer interaction, or AI systems, among others; it is clearly less similar to classical modeling work in deep learning. I see some researchers struggling with this shift. It means rather than owning the model, we have to think about the application, the workflow, the evaluations, the user feedback, and ultimately own the product rather than the model.
I believe that similar lessons will apply to many other domains, not only health. It also applies beyond academic research to companies building AI solutions in various domains.
The post Domain-Specific AI Should Focus on Workflows Rather Than Modeling appeared first on David Stutz.
