Harness scaling improves generalization of agentic reinforcement learning models
Harness Scaling for Generalizable Agentic Reinforcement Learning
Machine LearningArtificial IntelligenceComputation and Language
Summary
Using different ways to train AI agents to use tools and manage tasks helps them perform better in new situations. The paper shows that training these agents with many varied 'harnesses'—which are setups or frameworks for how the agents interact—makes the agents more flexible and able to handle unseen challenges. This approach, called Harness Scaling, trains models across a balanced mix of harnesses instead of just one fixed kind. Their experiments found this method improves how well models adapt to new, different environments and tools.
What this means in practice
- •For software engineers: Build AI systems that better adapt to new toolsets and interaction frameworks by training with diverse harness configurations.
- •For automation developers: Create more flexible automated agents that maintain performance when deployed in different environments or with varied external tools.
Authors
Fei Bai, Huatong Song, Chuan Hao, Ran Tao, Jian Yang, Bryan Dai, Wayne Xin Zhao, Ji-Rong Wen
Abstract
Agent harnesses have become a central component of modern agentic systems, shaping how models use tools, manage context, and coordinate long-horizon interaction. Realizing the benefits of these harnesses depends on how effectively the underlying model can make use of them, motivating reinforcement learning directly through the harness. However, existing approaches typically train through a single fixed harness, which can lead to specialization and limited transfer across harness configurations. To address this, we introduce \textbf{Harness Scaling}, a new scaling dimension for agentic RL that extends training from a single harness to diverse execution configurations. Concretely, we construct a factorized harness space, select a compact training subset with balanced factorial coverage, and organize cross-harness training to combine global coverage with controlled variation for the same task. Across multiple benchmarks, model scales, and RL algorithms, Harness Scaling consistently improves generalization to unseen white-box configurations. Moreover, models trained through structured white-box variation transfer effectively to unseen external black-box harnesses, outperforming models trained through a single complex black-box harness. These results establish harness diversity as a meaningful scaling dimension for agentic RL and highlight the importance of how this diversity is organized for cross-harness generalization.