Language agents learn to reduce external control during tasks
Harness Annealing: Learning to Act with Less External Control
Computation and Language
Summary
Many AI language agents depend on extra help to decide what to do next, like when to check answers or stop working. The authors explore whether these agents can learn to make those decisions themselves by gradually removing this help during training. They created a method that teaches models to take more control over their actions while still performing well. Their experiments show that bigger models can often work almost as well with less external guidance after learning this way.
What this means in practice
- •For ai product developers: Develop AI assistants that require less manual guidance by teaching them to manage their tasks independently.$Commercial implications: Enables creation of more autonomous AI assistants that reduce dependency on human-in-the-loop controls, improving scalability and user experience.
- •For automation engineers: Implement models that progressively reduce the need for external oversight during complex automated workflows.
Authors
Yingxuan Yang, Huacan Chai, Ying Wen
Abstract
Language agents rely on external harnesses to track state, organize workflows, and verify answers. Beyond providing tools and information, these harnesses supply control decisions about what to investigate, whether to revise, and when to stop. Training on successful harness-supported trajectories can improve task performance while leaving these decisions dependent on runtime intervention. We ask whether harness-supported experience can also teach the model to make these decisions, allowing the division of control to change as the model learns. We call this objective harness internalization: learning to assume specified control responsibilities while retaining task performance after the corresponding support is withdrawn. We introduce HARNESS ANNEALING TRAINING (HAT), which combines explicit control supervision with a curriculum over teacher trajectories collected under progressively weaker harnesses. Experiments with 9B and 35B models on SWE-QA and SWE-QA-Pro evaluate every checkpoint under four deployment harnesses. Selected annealed checkpoints operating with tools alone achieve scores close to those of their respective starting checkpoints deployed with the full harness. The benefits vary with model scale and deployment configuration, and further annealing does not uniformly improve performance. These findings suggest that harness-supported experience can help reduce the runtime control required by a trained agent.