Self-improving optimizers enhance AI agent workflows with tests
VERSE: Verified Self-Evolving Optimizer for Agent Harnesses
Artificial IntelligenceComputation and LanguageMachine Learning
Summary
AI systems that help other AI agents often use fixed tools to improve those agents. The authors find that if the optimizer itself learns to check its own changes by testing and verifying them, it can get better results. They created VERSE, a system where the optimizer tests edits, finds and fixes errors, and improves both its strategies and the agent's setup without changing the main AI model. This approach led to better performance on various tasks across different languages than other strong existing methods.
What this means in practice
- •For ai development teams: Improve automated AI agent setups by enabling their optimizers to self-test and refine workflows for better accuracy.
- •For multilingual software engineers: Enhance robustness of agent-driven tools across languages by adopting verification-backed optimizer self-improvement techniques.
Authors
Zekai Wang, Yingqiang Ge, Zekun Wang, Hai Wang, Yuhui Xu, Joshua Frandsen, Shancong Fu, Ashia C. Wilson, Chandan K. Reddy
Abstract
Harness evolution improves an LLM agent's prompts, tools, and workflow, while the optimizer's own tools and procedures often remain fixed. We study whether an optimizer can improve another agent more effectively by also improving how it diagnoses failures, develops edits, and tests their effects. Two observations guide our design. In a controlled study, optimizer self-evolution fails to improve performance without execution-based verification, but achieves the best result of that study when verification is available. Across five executors, self-evolving optimizers build their own tools for failure analysis, verification, training audits, and workflow control. Motivated by these findings, we introduce VERSE, a Verified Self-Evolving optimizer for agent harnesses. VERSE lets the optimizer test draft edits, replay failures, and perturb suspected steps before submission, while tracking fixes and regressions across rounds. Using this feedback, the optimizer revises both the executor harness and its own prompts, skills, tools, hooks, and notes, while the weights of the optimizer and executor models stay fixed. Under a shared protocol with disjoint training, validation, and test tasks, VERSE improves all four evaluated harness optimizers on held-out SWE-rebench tasks and newer out-of-distribution tasks in five languages. Its best validation-selected harness reaches 42.3% and 37.7% accuracy, respectively, against 39.2% and 29.3% for the strongest baselines. Code is available at https://github.com/wzekai/VERSE.