Software agents deleting old data can increase costs under some pricing models
The Hidden Costs of Forgetting: A Study of Trajectory Deletion in Software Engineering Agents
Software EngineeringArtificial Intelligence
Summary
When computer programs called software engineering agents forget or delete some of their past actions to save space, they might end up doing more work to get back the lost information. The researchers studied many ways to delete old info and found that deleting a lot at once works better if done after some delay. Surprisingly, this forgetting doesn’t always save money because some pricing models charge less when parts of the data are reused, and deleting breaks that reuse. The authors suggest that simply counting deleted text isn’t enough; one must consider how these agents behave and how costs are actually charged.
What this means in practice
- •For software development teams: Optimize the use of data deletion in AI-powered coding assistants to maintain their effectiveness while managing operational costs under realistic pricing.
- •For cloud service engineers: Design AI service cost models by accounting for caching effects when agents delete data to predict true savings and prevent unexpected cost increases.
Authors
Fernando Vallecillos-Ruiz, Islem Bouzenia, Michael Pradel, Leon Moonen
Abstract
Software engineering agents resend their entire trajectory to an LLM at every turn, motivating online trajectory reduction. Prior work shows that randomly deleting 75% of tokens in old turns may preserve agent effectiveness, but it remains unclear why, how agents compensate for missing content, and whether it actually reduces costs. This paper presents an empirical study of 42 trajectory-reduction configurations, focusing on effectiveness, agent behavior, and cost. We systematically explore deleting content from old turns by varying the scope, granularity, amount, and timing of deletion across 9,400 runs with two state-of-the-art agents on SWE-bench Verified. We find deleting large amounts of trajectory content (e.g., up to 100% of reasoning text) can preserve overall effectiveness. Importantly, such deletion must be applied with a delay, as immediate deletion tends to significantly degrade effectiveness. Agents often compensate for deleted content by re-fetching it, increasing both the number of turns taken and the input/output tokens. Whether trajectory reduction reduces costs depends on the pricing model: Under a flat pricing model, estimated costs fall in 26 of 42 evaluations. Surprisingly, under a caching-aware pricing model, zero evaluations yield lower costs, since reducing the trajectory breaks the cached prefix billed at a lower rate. Our findings have three implications: First, token counts alone are misleading; trajectory reduction techniques must consider overall costs under a realistic pricing model. Second, analyzing trajectories is crucial to understand agent behavior, as end-to-end effectiveness hides differences such as excessive re-fetching. Finally, these results highlight the need for future work on memory management to reduce costs without sacrificing effectiveness.