Medical system routes questions to expert ai models for better answers
MedRouter: Demystifying Knowledge Differences Across Medical LLMs for Routing-Based Reasoning
Computation and LanguageArtificial Intelligence
Summary
Answering medical questions is tricky since different AI models specialize in different areas of medicine. The authors created MedRouter, a system that decides which specialized AI to ask based on the question, then combines their answers for a clearer, more accurate response. They trained the system in two steps to recognize each specialist’s strengths and improve which ones it chooses. Tests on various medical question sets showed MedRouter gave better answers than other routing methods, making medical question answering more reliable.
What this means in practice
- •For medical ai developers: Build systems that direct medical questions to the best specialized AI model for more accurate multimodal answers.
- •For customer support teams: Integrate routing AI to improve handling of specialized queries by sending them to the ideal expert model.
Authors
Lang Cao, Binghang Lu, Yuhao Shen, Yue Guo
Abstract
Medical question answering spans diverse specialties and modalities, and individual medical large language models (LLMs) exhibit distinct strengths across tasks and domains. This heterogeneity suggests that combining specialists may enable broader coverage of medical questions than relying on any single model. However, existing LLM routing methods primarily seek to balance answer quality and inference cost, leaving open how to exploit differences in specialist competence to improve medical reasoning. In this paper, we introduce MedRouter, an agentic system that uses an embedding-based multi-label router to select and query specialist LLMs, then passes their responses to a generator to produce the final answer. We further propose SCALE (Specialist Competence-Aware Learning), a two-stage training framework that first trains the Router with specialist correctness supervision and then optimizes its selections through reinforcement learning. The second stage uses a Performance Gain Reward (PGR) that measures how specialist information affects the generator's answer correctness relative to answering without that information. Experiments on eight text-based and multimodal medical QA benchmarks show that MedRouter outperforms the strongest routing baseline by 8% in average accuracy. Our analysis of specialist outputs further reveals distinct strengths and complementary question-level coverage, motivating learned routing to combine these capabilities for more comprehensive medical reasoning.