TraceLab: Characterizing Coding Agent Workloads for LLM Serving
2026-06-29 • Machine Learning
Machine LearningArtificial IntelligencePerformance
AI summaryⓘ
The authors collected data from about 4,300 real coding-agent sessions using two popular models to better understand how these AI coding tools are actually used. Their analysis found that the agents work in long stretches with lots of context but short responses, and they call many different tools with patterns that are uneven. Based on this, the authors suggest improvements for making the systems faster and more efficient, like smarter ways to manage tool calls and memory. They also shared their dataset and tools publicly for others to study and build on.
coding agentslarge language models (LLMs)workload analysistool callsprefix cacheKV-cache managementfill prefetchingautonomous loopsmodel serving
Authors
Kan Zhu, Mathew Jacob, Chenxi Ma, Yi Pan, Stephanie Wang, Arvind Krishnamurthy, Baris Kasikci
Abstract
Coding agents are rapidly becoming a major application of agentic LLMs, but serving them efficiently remains challenging. Progress on this challenge requires understanding real workload patterns, yet the data needed for such analysis is largely absent. Existing public traces and benchmarks do not capture real, day-to-day coding-agent usage across multiple agents and model families for serving-system analysis. To help fill this gap, we collect and release a trace of roughly 4,300 coding-agent sessions, containing about 350,000 LLM steps and 430,000 tool calls from our own day-to-day use of Claude Code and Codex. Our analysis shows that coding-agent workloads feature long autonomous loops, long contexts with short outputs, diverse and heavily-tailed tool calls, and high but imperfect prefix cache hit rates. These findings point to concrete opportunities for optimizing serving, including lower-overhead tool calling, append-length-aware prefill, semantic-aware tool-latency prediction, and improved KV-cache management around human-paced gaps. We release the dataset, trace collection pipeline, and analysis code at https://github.com/uw-syfi/TraceLab.git; the project website is https://tracelab.cs.washington.edu.