Coding agents struggle to optimize real microservice system performance
Benchmarking Coding Agents for System-Level Performance Optimization
Software Engineering
Summary
Fixing software performance is hard when apps run many small services that must work well together. The authors created a benchmark called MicroPerf-Bench to test how well coding agents can improve overall system speed by fixing known problems spread across services. They found that these agents often make correct code changes that do not actually improve performance, showing that understanding where bottlenecks happen in complex systems is tough for current tools.
What this means in practice
- •For software engineering teams: Evaluate and improve coding agent tools aimed at optimizing real-world microservice system performance end-to-end.
- •For cloud infrastructure operators: Test automated fixes to performance problems that emerge from interactions in distributed services running on cloud platforms.
Authors
Yanze Zhang, Maggie Yongqi Guan, Shudong Liu, Qi Hao, Kanye Ye Wang, Meng Qu, Xiaobo Zhou
Abstract
Coding agents are becoming an increasingly important paradigm for automating software engineering tasks. However, existing coding benchmarks primarily evaluate agents at the function or repository level, leaving their capabilities in more complex distributed software systems largely unexplored. In practice, many modern applications adopt microservice architectures, where end-to-end performance emerges from interactions among multiple independently running services, requiring agents to reason beyond local code changes. To study this capability, we introduce MicroPerf-Bench, a benchmark for evaluating coding agents on end-to-end performance optimization in microservice systems. Constructing such a benchmark is challenging because reproducible system-level performance fixes are sparse and difficult to attribute. We address this challenge by injecting empirically grounded performance anti-patterns into real-world open-source systems while preserving functionality and ensuring measurable performance degradation. The resulting benchmark contains 65 optimization instances across seven systems and nine anti-patterns, each with an executable environment, functional tests, and mixed-workload evaluation. Across 1,755 runs covering nine agent-model configurations, the best configuration recovers only 42.0% of the injected performance degradation. The dominant failure is not invalid code, but functionally correct and constraint-compliant changes that fail to improve end-to-end performance, highlighting system-level bottleneck localization as a key challenge for current coding agents. The source code and data are available at https://github.com/zyzcjnb/microperf.