Model aware tool learning improves language model tool use accuracy
MATCH: Model-Aware Tool Learning with Curriculum Scheduling and Hierarchically Gated Rewards
Machine LearningArtificial IntelligenceComputation and Language
Summary
Large language models sometimes need to use external tools to get things done, but figuring out when and how to call these tools well is tricky. The authors present MATCH, a method that helps the model learn tool use better by adjusting practice problems based on what the model is ready to learn and carefully giving rewards only when the model gets parts of the tool call correct. This new way improves how accurately the model calls tools compared to past methods, tested on two different tool-using tasks.
What this means in practice
- •For software development teams: Enhance language model assistants that automate API calls by better training them to use tools accurately across evolving tasks.
- •For cloud platform operators: Improve management of tool-using AI services by integrating adaptive curricula and reward systems that align with model learning progress.
Authors
Shihao Liu, Hao Yin, Lijun Liu, Zhengzong Chen, Yuanyuan Zhao, Fei Huang
Abstract
Tool learning enables large language models (LLMs) to use external tools for tasks beyond parametric knowledge. Reinforcement learning can optimize tool-call behavior from feedback, but current methods still face two problems: fixed-threshold curricula can become misaligned with the policy's evolving capability boundary, and additive rewards can leak argument-level credit when the predicted tool is wrong. To address these problems, we propose MATCH, a closed-loop framework for model-aware tool learning with curriculum scheduling and hierarchically gated rewards. Model-Aware Curriculum Learning (MACL) maintains reward-derived sample difficulty that co-evolves with the policy, and each epoch selects samples near the current capability boundary together with a top-k pool of harder cases. Hierarchical Tool-call Gated Reward (HTGR) scores tool name, argument key, and argument value as a gated chain, granting credit at each level only when prerequisites hold. The same HTGR rewards drive both GRPO updates and MACL's difficulty refresh, closing the loop between policy optimization and sample scheduling. On API-Bank and BFCL V3, MATCH reaches 72.19% and 62.87% overall accuracy, outperforming the main supervised and RL-based baselines. Backbone experiments further show consistent improvements across four backbones from two model families.