VDAR-Router: Adaptive LLMs Routing via Verbalized Query Difficulty Analysis Retrieval
2026-07-20 • Computation and Language
Computation and Language
AI summaryⓘ
The authors introduce VDAR-Router, a method to choose the best large language model (LLM) for each question by understanding how hard the question is. Instead of just looking at the question's meaning, their method first analyzes difficulty and then finds similar past questions with known results. Using this information, it picks the model that gives the best balance of good answers and low cost. Tests show VDAR-Router works better than older methods without needing extra training. The authors also show that explaining the difficulty helps find better examples and makes smarter decisions.
large language modelsmodel selectionroutingquery difficultyretrieval-based methodscost-performance trade-offsimilarity searchreward functionzero-shot routingcase study
Authors
Yu-Chien Tang, Jun-Chen Hung, Wen-Chih Peng, An-Zi Yen
Abstract
Large language models are increasingly used in practical systems, making efficient model selection important for reducing deployment cost. LLM routing has emerged as a practical solution for allocating each input query to an appropriate model under a desired cost-performance trade-off. Existing routing methods often estimate model suitability from the surface semantics or embedding similarity of the input query. However, such methods may ignore the underlying difficulty of a query, leading to suboptimal routing decisions. To address the challenge, we propose VDAR-Router, a difficulty-aware retrieval-based routing framework. For each input query, VDAR-Router first generates an explicit difficulty analysis. It then retrieves historical examples with similar difficulty profiles. Based on the retrieved records, it estimates candidate model suitability and selects the model using a reward function that considers both performance and cost. Experiments on three datasets show that VDAR-Router consistently achieves better cost-performance trade-offs than existing baselines. These results demonstrate the effectiveness of difficulty-aware retrieval for training-free LLM routing. Case studies further show that explicit query analysis helps retrieve more relevant examples and supports more reliable routing decisions.