When is Routing Meaningful? Diversity and Robustness in Language Model Societies
2026-07-10 • Multiagent Systems
Multiagent Systems
AI summaryⓘ
The authors explain that when using multiple AI models together, it's not enough to just look at how accurate or fast they are. They say it's important that different models behave differently and that the way queries are sent to them stays consistent even if the question wording changes. They created new ways to measure these ideas and tested them on existing systems. Their findings show that having a small, well-chosen group of models works almost as well as a big group and that some routing methods are accurate but not stable, meaning they don’t always assign similar questions to the same model.
multi-model systemsrouting policybehavioral differentiationrouting stabilityHierarchic Social Entropy (HSE)perturbation robustnessEmbedLLMRouterBenchKNN routersprompted routing
Authors
Fantine Huot, Michael Kaisers, Mirella Lapata
Abstract
Routing policies for multi-model systems are evaluated almost exclusively on task accuracy and inference cost. We argue that two properties, orthogonal to performance, determine whether routing is meaningful. First, the society of actors must be behaviourally differentiated: if all actors respond identically, routing is vacuous. Second, the routing policy must be stable: surface-form variants of a query should be assigned to the same actor. High task accuracy is compatible with violating both properties, since a router can operate over a redundant society or assign queries inconsistently, preventing specialisation regardless of performance. We adapt Hierarchic Social Entropy (HSE) to language-model societies and introduce a perturbation-based robustness metric to diagnose these failure modes. Applied to EmbedLLM and RouterBench, we find that HSE exhibits strong diminishing returns, suggesting that a curated subset of fewer than ten agents recovers most available diversity in a large pool -- a practical coreset heuristic for society design. We further find that KNN routers gain accuracy from specialist societies but collapse in robustness under perturbation, while prompted routing remains stable across all perturbation types -- illustrating that accuracy and meaningfulness can sharply diverge.