Coding agents waste money with inefficient behaviors and improvements vary
Analyzing and Mitigating Cost-Inefficient Behaviors in Coding Agents
Artificial IntelligenceSoftware Engineering
Summary
Coding agents help write software but often waste money in how they work. The authors studied behaviors that cause this waste, like repeating searches or rerunning tests unnecessarily. They tested ways to fix these problems, finding that developer-designed skills worked best to save costs. Agent-made skills and smarter retrieval didn’t always help and sometimes made things worse. This shows better ways to make coding agents cheaper and more efficient.
What this means in practice
- •For software engineering teams: Reduce expenses in automated coding workflows by applying developer-designed guidance to limit costly redundant operations.
- •For automation tool developers: Improve cost efficiency of coding assistant tools by integrating structure-aware retrieval and custom skill modules based on these findings.
Authors
Yiran Hu, Nan Jiang, Shanchao Liang, Anik Dey, Yi Wu, Lin Tan
Abstract
Although effective, coding agents often incur substantial monetary costs. Their recurring cost-inefficient behaviors remain underexplored. We conduct the first study of behavioral cost inefficiencies in coding agents, analyzing 1,200 trajectories from Claude Code and Mini-SWE-Agent across four configurations on SWE-bench Verified. We identify three cost-inefficient behaviors: subsumed retrieval, similar script generation, and test re-execution. We then evaluate three mitigation strategies: structure-aware retrieval, agent-synthesized skills, and developer-designed skills, over 10k trajectories on held-out SWE-bench Verified and Pro tasks. Our main findings are: (1) The three behaviors affect 79.00\%--98.00\% of coding tasks and account for up to 22.75\% of task cost. (2) Structure-aware retrieval can introduce retrieval overhead and alter agent delegation, causing inconsistent improvements in retrieval efficiency and cost increases of up to 28.14\%. (3) Agent-synthesized skills tend to produce low-level, trace-specific guidance, limiting their effectiveness and generality. (4) In contrast, developer-designed skills provide high-level, trace-agnostic guidance, reducing cost by up to 41.73\%, roughly twice the maximum gain from agent-synthesized skills.