Papers for

software platform builders

Papers whose findings have a practical use for this group, as judged from the abstract. Open a paper to read what it means in practice.

Scaling data diversity improves large language model tool use performance

Scaling Laws for Tool-Use Agents via Controllable Environment Synthesis

Abstract: We explore how to build performant large language model (LLM) tool-use agents that complete complex, multi-step tasks. Specifically, we study how data composition in synthetically generated tool-use environments affects the training of such agents. To do so, we propose the SynthAPI pipeline, which explicitly controls the diversity of generated environments along the axes of tools, tasks, and the databases on which the tools operate. By systematically varying SynthAPI's generation controls in 458,752 tasks across 849 trained models of four different sizes, we derive a scaling law that predicts downstream tool-use agent generalization based on the scale and composition of the synthesized environments. These predictable relationships reveal several lessons for scaling tool-use agent data. As data budgets grow, scaling tool and database diversity becomes more effective than increasing the number of tasks, and increasing data complexity yields larger gains. Scaling according to these findings yields agents that outperform models trained on data from prior work in large-scale tool-use agent and synthetic environment generation across four evaluation benchmarks.

Tue 27 OctMachine Learning
The gist
Making AI agents better at using tools to solve complicated tasks depends on the kind of training data they see. The authors created a way to control the variety of tools, tasks, and databases in synthetic environments for training. They found that as training data grows, increasing the variety of tools and databases helps more than just adding more tasks. Their findings guide how to build stronger AI agents that use tools across different challenges.
Open → 2610.37648v1