Harness system optimizations cut AI agent costs without hurting accuracy
System Optimizations for Cost-Efficient Agent Harnesses
Artificial Intelligence
Summary
AI agents use language models (LLMs) to perform tasks, but the way the agent system is built can affect how much it costs to run these tasks. The authors found that changing the system around the LLM doesn’t change how well the tasks get done but can save a lot on costs. They created a platform called Agency that lets developers add cost-saving improvements easily to existing AI agent setups. Their tests show these system tweaks cut costs by up to 35% without making the agents any less accurate.
What this means in practice
- •For ai application developers: Reduce costs of AI agents running on language models by implementing harness-side optimizations using the Agency platform.
- •For cloud infrastructure teams: Lower operational expenses by integrating system-level improvements that decrease token usage and LLM calls without changing AI model deployments.
Authors
Jongseok Park, Jiaxiang Yu, Eric Zhou, Tony Hong, Sunga Kim, Yusuf Afifi, Ian Wong, Ion Stoica, Alvin Cheung
Abstract
AI agent harnesses play the critical role of providing LLMs with an interface to a computing environment. Recent studies show that AI agents' task-execution accuracy depends mostly on the LLM used, while harness-side variances such as system prompts or tools have minimal impact. Rather, such harness-side variances show a larger impact on the cost of agent executions, achieving cost reductions up to 2.6 times on the same model and task. To further explore these harness-side cost optimization opportunities, we analyze the cost accumulation patterns across various agent traces and identify system opportunities to reduce token use and LLM calls without modifying the model or affecting task accuracy. Based on this analysis, we design Agency, a harness interposition platform that provides a unified abstraction and allows easy and portable implementation of harness-side system optimizations on existing off-the-shelf harnesses such as Claude Code or OpenAI Codex. Using Agency, we implement proof-of-concept optimization techniques based on our analysis and achieve cost reductions up to 35% across various real-world benchmarks on existing harnesses without compromising accuracy.