Harness-R1: Learning to Edit Executable Runtime Harnesses from Agent Failure Trajectories
2026-08-03 • Artificial Intelligence
Artificial Intelligence
AI summaryⓘ
The authors propose a new method called Harness-R1 that helps improve how AI agents built with large language models fix their own mistakes during tasks. Instead of just updating the AI model itself, Harness-R1 trains a special 'harness engineer' to edit and improve the AI agent’s runtime behavior based on past failures. This harness engineer learns through trial and error to make better fixes that actually lead to success. Tests show that this approach improves task success rates across several benchmarks. The authors suggest that both the AI agent and its harness engineer can be improved together over time.
large language modelsagent harnessruntime editingreinforcement learningpolicy optimizationpost-trainingfailure-conditioned learningexecutable patchesfine-tuningtask success rate
Authors
Shuai Shao, Kangning Zhang, Qingyao Li, Shijian Wang, Hao Wang, Wenxiang Jiao, Yuan Lu, Yi Guo, Weiwen Liu, Weinan Zhang
Abstract
Agents built around large language models continually accumulate interaction trajectories during deployment, yet their behavior typically remains fixed. Beyond updating model weights, these trajectories can improve the agent harness that constructs context, mediates tools, validates actions, and recovers execution. We introduce Harness-R1, the first method, to our knowledge, that makes failure-conditioned, lifecycle-wide editing of an existing executable runtime a learned capability. It post-trains a dedicated harness engineer with online reinforcement learning so that its edits are optimized for the realized task success they produce, rather than proposed by a fixed editor. A separate 9B engineer converts batches of target-agent failures into validated executable patches; fresh same-batch reruns of the frozen target provide outcome rewards, so training updates only the engineer. Cold-start supervised fine-tuning initializes this editing policy, which is then trained online with group-relative policy optimization. Across WebShop, ALFWorld, and DBBench, Harness-R1 raises vanilla Qwen3.5-9B success from 44.3% to 53.6% (+9.3 percentage points). After direct target-agent fine-tuning, a target-specific engineer raises the average further from 59.2% to 64.2% (+5.0 points); because these gains hold both before and after fine-tuning the target, Harness-R1 points toward co-evolving the harness engineer and the target agent.