Turbo harness optimizes task execution models for each instance
Turbo Harness: Instance-Adaptive Harness Optimization
Artificial Intelligence
Summary
Finding a single best setup to solve many different tasks can be hard because what works well on average might not be best for every situation. The authors introduce Turbo Harness, a way to start with a good general setup and then tweak it for each specific task using lessons learned from earlier attempts. Turbo Harness creates a guide to these lessons and uses it to fix the general setup so it fits each new task better. Tests show this method does better than older approaches across various tasks that involve interacting with systems and long-term problem solving.
What this means in practice
- •For interactive system developers: Create task-specific ways to run agent models that improve performance for each unique interaction scenario.
- •For software engineering teams: Generate customized harnesses for different software tasks to enhance development and testing workflows.
Authors
Tunyu Zhang, Hao Wang, Kai Xu, Dimitris N. Metaxas
Abstract
Automating the search for effective harnesses is an important step toward enabling agents to recursively self-improve. Existing harness optimizations typically produce a single global harness that is applied uniformly across task instances. However, a harness that works well on average may not be optimal for every instance. We introduce Turbo Harness, a framework that can adapt a globally optimized harness to each instance by reusing information generated during the original optimization process. Specifically, Turbo Harness recycles artifacts produced during a completed global harness optimization run, and summarizes them into a structured playbook. We train a harness editor to leverage this prior optimization experience to generate instance-specific patches to the global harness. At inference time, the editor uses the instance and the playbook to construct a tailored harness in which the execution model operates. Through numerical experiments, we show that Turbo Harness consistently outperforms existing harness optimization baselines across seven benchmarks spanning interactive agent tasks, software engineering, and long-horizon terminal tasks.