Small language models ask large models key questions to save cost
Learning to Ask: Information Acquisition for SLM-LLM Collaboration, under a budget
Artificial Intelligence
Summary
Big language models are very good at complex thinking but expensive to run, while small models are cheaper but not as smart. The authors propose a method where the small model does most of the thinking and only asks the big model for help when it really needs information. Their system learns when to ask, what questions to ask, and how to use the answers effectively. This method reduces cost while keeping performance high and works across different tasks and advisor models.
What this means in practice
- •For software developers: Save computing costs by enabling small models to selectively ask large models for key reasoning steps when working on code tasks.
- •For product managers in ai services: Balance cost and performance in AI products by incorporating frameworks that optimize when to query expensive large language model APIs.$Commercial implications: Supports developing cost-efficient AI applications that integrate multiple language model sizes for more affordable, smarter services.
Authors
Yongjun Kim, Xiaoxiao Li, Jaeho Lee
Abstract
Collaboration between a small language model (SLM) and a large language model (LLM) offers an opportunity to combine the efficiency of smaller models with the strong reasoning capabilities of larger ones. Existing approaches primarily frame such collaboration as a computation allocation problem, determining which model should handle each portion of the reasoning process. In black-box API-based settings, however, this paradigm can be inefficient due to coarse-grained delegation or repeated transmission of context across model switches. In this work, we instead formulate SLM-LLM collaboration as an information acquisition problem, under an API budget constraint. The SLM remains the primary reasoner and selectively queries a black-box LLM advisor only when needed, issuing targeted queries rather than delegating the reasoning process itself. To realize this strategy, we develop a three-stage RLVR framework that learns whether to call the advisor, how to formulate useful queries, and how to integrate the collaboration into the reasoning process by jointly refining advisor invocation and information use. Across mathematical reasoning and coding tasks, our approach improves the performance--cost tradeoff over existing collaboration baselines and, in some settings, matches or exceeds oracle problem-level routing. Finally, we show that our strategy can transfer to other advisor model families, without further training.