Training language agents with better task sequences in complex tool environments
GRADUS: Discovering and Recovering Agent Lessons from the Environment's Latent Syllabus
Artificial Intelligence
Summary
Some AI systems need to learn how to use software tools that have many rules, like airline booking and refund systems. The authors designed Gradus, a method that figures out the hidden steps and conditions behind the rules of such environments without needing example tasks first. Gradus creates a series of tasks that get harder and checks how well the AI did, using that feedback to improve both the tasks and the solutions. This approach helps train AI agents to understand and follow complex policies better, raising their accuracy significantly.
What this means in practice
- •For enterprise software developers: Train AI agents to reliably operate complex business tool systems that enforce multiple combined policies, improving automation accuracy.
- •For automation platform engineers: Create adaptive training curricula for AI agents that interact with integrated tool environments, enhancing performance without manual task design.
Authors
Qing Ping, Wei Yang, Panpan Xu, Johannes Kirmayr, Lukas Stappen, Junjie Tang, Jun Huan
Abstract
Training LLM agents for policy-constrained tool environments, such as an airline backend with booking, refund, and cancellation policies, requires tasks that exercise the environment's rules individually and in combination. Existing synthesis pipelines let a model propose tasks and retain only those it immediately solves, biasing training data toward easier behaviors. We introduce Gradus (Graded Rungs of Ascending Difficulty from an Underlying System), which decouples what a task should teach from the concrete task and solution used to teach it. Without seed tasks, Gradus treats a fixed environment as a latent syllabus, extracting guarded behavioral operators that capture how actions depend on current conditions and how their effects reshape subsequent decisions. It composes these operators into increasingly demanding tasks while using corpus-level feedback to target missing outcome combinations. Crucially, Gradus jointly calibrates tasks and solutions against the environment: execution feedback identifies which side to refine while preserving the intended behavioral requirements. Verification thus becomes a recovery signal rather than a terminal filter, coupling coverage-guided exploration with the preservation of difficult examples. From off-the-shelf backbones, the resulting curriculum lifts a 30B model from 59% to 89% on $τ^2$-Bench and from 47% to 83% on CAR-Bench with an Opus 4.5 proposer, and to 76% and 62%, respectively, with an open-weight Qwen3-235B proposer. These results show the value of jointly controlling which behaviors are explored and which survive into verified training data.