Improving language model agents by alternating training and task guidance updates
VACE: Validation-Gated Alternating Co-Evolution of Agent Models and Harnesses
Machine LearningArtificial IntelligenceComputation and Language
Summary
Language model agents can be improved not just by changing their internal parameters but also by improving how they are guided during tasks. The authors propose a method called VACE that alternates between training the model and improving its guidance system, called a harness. After each training phase, they check if the harness update actually helps using a validation step before continuing. This approach gave better results on certain question answering and automation benchmarks compared to just training the model or updating the harness without checks.
What this means in practice
- •For ai system developers: Improve agent performance in AI systems by alternately refining model weights and task guidance using validation-gated updates.
- •For automation platform engineers: Enhance automation task accuracy by integrating alternating co-evolution of agent models and execution harnesses with validation checks.
Authors
Jiexing Qi, Yu He, Jun Liu, Qichen Huang, Shaohua Hu, Zhan Dang, Guohua Chen, Rui Yang, Wen Jiang, Yang Liu, Tao Lyu, Fangming Li
Abstract
Language model agents can be improved by updating their model weights or refining the harness that guides task execution. These components are coupled: weight updates change how the model uses the harness, while harness updates change the trajectories used for training. We propose VACE, Validation-Gated Alternating CoEvolution, which alternates agentic reinforcement learning with trajectory-driven harness refinement. After each RL stage, VACE reuses the collected trajectories to propose a harness revision and evaluates the incumbent and candidate with the updated model held fixed. The candidate guides subsequent training only if it improves validation performance. With Qwen3.5-9B, VACE achieves 45.26% test accuracy on OfficeQA and a mean partial-credit score of 75.19% on AutomationBench, exceeding weight-only RL by 6.43 and 9.09 percentage points and ungated alternation by 4.59 and 6.95 points, respectively. Across 44 harness proposals, 17 reduce validation performance at the updated checkpoint and are rejected before subsequent RL training, highlighting the importance of validation gating.