Object-oriented harnesses improve language model task handling
Object-Oriented Harness: Evolving Agent Harnesses through Object-Oriented Design
Computation and LanguageSoftware Engineering
Summary
Language models often need extra code, called harnesses, to guide how they use prompts and tools for different tasks. A single harness might not work well for all kinds of tasks that need different handling. The authors created a way to evolve many specialized harnesses arranged in a hierarchy using object-oriented programming ideas, so each task can use the best one. Their system learned how to pick the right harness for each task and was tested on different complex benchmarks, showing better results than simpler methods.
What this means in practice
- •For workflow automation teams: Create specialized executable harnesses that improve handling of diverse and complex business workflows using language models.
- •For software developers integrating ai tools: Use a harness routing system to select optimized control flows for different workspace tasks involving language model tools.
Authors
Gaotang Li, Bhavana Dalvi Mishra, Yifei Ming, Yanfei Chen, Zhongying CuiZhu, Lesly Miculicich, Long T. Le, Rui Meng, Chun-Liang Li, Hanghang Tong, Chen-Yu Lee, Tomas Pfister
Abstract
Agent harnesses are executable code that specifies the prompts, tools, and control flow around a language model. A single global harness can struggle with heterogeneous tasks that require different execution strategies. In this work, we introduce Object-Oriented Harness (OO-Harness), a harness optimization framework that evolves a hierarchy of executable harness classes using object-oriented design. During training, an evolution agent uses execution records to create a family of specialized harnesses, each implemented as a subclass of the base or an existing specialized harness via inheritance. At test time, a harness router uses routing lessons accumulated during training to select a specific harness for each task. Across three long-horizon benchmarks spanning business workflows, workspace tasks, and tool use, OO-Harness achieves the best performance among the compared methods, with an absolute improvement of up to 15% over the base. Extensive ablation studies examine the contributions of its core components.