Recursive Harness Self-Improvement
Machine LearningArtificial Intelligence
Summary
The authors explore a way to improve how programs called harnesses guide AI agents during tasks. Instead of only using harnesses to run agents, they suggest harnesses can also help train future AI by producing better data. They introduce Recursive Harness Self-Improvement (RHI), a method that helps users tweak harnesses iteratively to boost agent performance and reduce computational cost. Their experiments show that a few rounds of RHI make agents work better by managing information flow more effectively, not just by thinking longer. They also propose a theory explaining why RHI works, highlighting its potential for ongoing AI improvement.
Authors
Hyunin Lee, Jinglue Xu, Jeffrey Seely, Donghyun Lee, Matei Zaharia, Yujin Tang
Abstract
Under model--harness co-evolution, harnesses are not merely inference-time scaffolds but data-generating components whose execution traces can shape future foundation models. This motivates harness-in-the-loop learning: optimizing harnesses for both immediate agent performance and the quality of traces used for future model training. However, continually updating provider-built scaffolds is costly and labor-intensive. We therefore investigate whether optimizing user-constructed harnesses in a task-specific manner can improve execution-trace quality while remaining computationally lightweight and requiring only a few update iterations. To this end, we introduce Recursive Harness Self-Improvement (RHI), which represents the harness as a prompt-level specification of the agent loop and iteratively refines it using pairwise feedback over its own revision history. Across 30 synthetic machine-learning research tasks spanning quantitative finance, robotics, and pharmacy, a few RHI iterations suffice to substantially raise the performance ceiling of low-reasoning-effort agents, exceeding the corresponding maximum-reasoning-effort setting while reducing inference cost by up to 60%. We show that these gains arise primarily from improved task-specific context management through more effective inter-agent information flow rather than longer reasoning traces. Finally, we formalize this behavior as an information-theoretic hypothesis for RHI's implicit optimization objective, suggesting RHI as a practical algorithm for continual learning within the paradigm of model--harness co-evolution.