Cliffcompaction cuts costs for long context coding agents
CliffCompaction: Cost-Efficient Compaction for Long-Horizon Coding Agents
Artificial IntelligenceMachine LearningSoftware Engineering
Summary
Many AI agents need to remember lots of information to solve complex problems but have limited space to keep it all. The authors created CliffCompaction, a way to squeeze that memory down by throwing out or cutting parts without rewriting anything, which saves money without losing performance. This method avoids building errors by never compacting already compacted info, allowing the AI to learn continuously over long periods. Their approach speeds up coding tasks and beats existing specialized methods while working broadly across different systems.
What this means in practice
- •For software developers: Reduce computation costs and latency when running AI coding assistants with long context needs using CliffCompaction's method.
- •For cloud service operators: Offer more cost-efficient AI services by integrating CliffCompaction to manage agents scaling over extended sessions.
Authors
Trang Nguyen, Eulrang Cho, Bingqing Chen, Tim Dettmers
Abstract
Agents often work on complex problems that require millions of tokens of context, which necessitates compacting across sessions due to limited context windows. We develop CliffCompaction, an autocompaction technique that reduces cost by up to 50% under a bounded context while maintaining or improving performance on Terminal-Bench and achieving new levels of efficiency for test-time scaling and state-of-the-art results on KernelBench. The per-rollout savings of CliffCompaction make the performance--cost trade-off of test-time scaling more efficient, adding over 10 percentage points on Terminal-Bench for less than the cost of two full-context runs. Under parallel test-time scaling, CliffCompaction lets Kimi K2.6 match Opus 4.7, and exceed Opus 4.6 and GPT-5.3 Codex at lower cost. The key to CliffCompaction's effectiveness is that it keeps compacted information faithful by only truncating or dropping content, never rephrasing or rewriting it. We never compact a compaction---each pass operates only on original content, and prior compacted output is discarded, preventing context drift from accumulating. These properties sustain continual learning over sessions exceeding a million tokens: on KernelBench, CliffCompaction reaches CUDA kernel speedups of $2.23\times$ after 200 steps and $3.58\times$ after 400 steps, surpassing specialized search algorithms and trained agents despite being a general-purpose compaction technique. We open-source a scaffold-agnostic API-proxy implementation of CliffCompaction usable with Claude Code, Codex and other harnesses.