Controlled model collaboration improves multi-LLM answers efficiently
COMED: The Missing Middle Between Routing and Collaboration in Multi-LLM Inference
Computation and LanguageArtificial Intelligence
Summary
Large language models (LLMs) often struggle to answer every question reliably on their own. The authors found that combining LLM answers all the time can sometimes help but can also make some correct answers worse. They introduce a method called COMED that decides when to trust one model’s answer and when to ask others to help, improving accuracy while avoiding unnecessary extra work. This approach works better than just picking one model or always combining answers, especially in medical and scientific tasks.
What this means in practice
- •For medical ai teams: Improve diagnostic question answering by selectively combining multiple LLMs to increase accuracy without heavy computation.
- •For enterprise ai developers: Deploy multi-LLM systems that balance response quality and efficiency by escalating queries only when collaboration adds value.
Authors
Norah Alballa, Wenxuan Zhang, Salma Kharrat, Fares Fourati, Zafar Ayyub Qazi, Mohamed Elhoseiny, Marco Canini
Abstract
No single Large Language Model (LLM) is uniformly reliable across queries, motivating multi-model inference systems that either route among models or combine their outputs. However, routing stops after selecting an initial model, while dense collaboration invokes peers on every query. We show that collaboration is non-monotonic: peers can recover failures that no model solves alone, but can also corrupt initially correct answers. We introduce COMED (Controlled Model Escalation for Multi-LLM Deliberation), a post-anchor controller for selective cross-model collaboration. COMED uses anchor self-consistency, router margin, and a lightweight peer probe to accept confident answers, verify ambiguous cases, and escalate only when collaboration is likely beneficial. We formalize this trade-off with a rescue-harm decomposition showing that selective collaboration improves when rescued errors outweigh collaboration-induced harms. Across medical, scientific, and general reasoning benchmarks, COMED improves fixed and routed anchors in all 16 open-weight settings, with gains up to +10.7 percentage points on MedQA while invoking fewer models and using fewer decoded tokens than dense collaboration. On HLE with frontier models, COMED improves GPT-5.5 from 23.1% to 28.1%, outperforming dense collaboration and achieving the best results.