Small open language model selects tools better than giant models
Training Small Language Models for Transferable Tool Selection
Machine LearningArtificial Intelligence
Summary
When computer programs use many tools, picking the right one for a complicated instruction is tricky because descriptions are too long to read all at once. The authors found a way to train a smaller and open language model to choose tools as well or better than much bigger commercial models. Their model learns by trial and error and performs well even when asked to work with new tools or different search methods. This makes it easier to build smarter programs that decide which tool to use without relying on huge proprietary systems.
What this means in practice
- •For software developers: Develop language models that effectively select appropriate tools for complex user requests across various software ecosystems.
- •For cloud platform operators: Improve service offerings by integrating smaller, efficient models that can handle tool selection without large-scale commercial LLMs.$Commercial implications: Enables cost-effective deployment of AI-powered tool selection services competing with giants like ChatGPT for cloud customers.
Authors
Zhaoyi Zhou, Akshayaa Magesh, Jeongyeol Kwon, Artem Kuriksha, Daniel Jiang, Andrea Zanette, Ayush Jain
Abstract
As LLM agents operate over corpora of more than ten thousand tools, selecting the right tool for a complex multi-step user instruction becomes a central bottleneck. Tool descriptions cannot all fit in context, and relevance-based retrievers only rank tools without knowing which are actually needed. Recent work instead leverages proprietary commercial LLMs like Claude or ChatGPT to drive a retriever, decomposing the instruction and selecting over retrieved candidates. We show that a small, open model can be trained via reinforcement learning to perform the same functionality. On ToolBench, a Qwen3-8B model post-trained with our recipe outperforms Llama-4-Maverick (402B). In addition, the trained model transfers directly to different retrievers and generalizes to new tool corpora in different domains.