Retain or Consolidate? Budget-Dependent Operator Selection for Language Agent Memory
2026-07-20 • Artificial Intelligence
Artificial Intelligence
AI summaryⓘ
The authors study how language-based agents use memory, which is limited by model size and cost. They analyze two ways to manage memory: keeping exact details (retention) or compressing information (consolidation). They introduce a method to decide when to use each approach based on available memory and show that consolidation works better under tight memory limits while retention is better when there is more space. Their experiments on two datasets confirm these findings, also showing that some ways of consolidating information work better than others.
language agentslarge language modelsmemory retentionmemory consolidationcontext windowcompressionutility estimationbenchmark evaluationevidence coverage
Authors
Qingcan Kang, Mingyang Liu, Shixiong Kai, Kaichao Liang, Zhentao Tang, Yuqi Cui, Tao Zhong, Mingxuan Yuan
Abstract
Language agents depend on memory across interactions. However, the limited context windows of large language models (LLMs) and their inference costs constrain how much memory can be used at once. Existing systems mainly follow two strategies: memory retention and memory consolidation. Retention keeps raw records and preserves exact details, but relevant evidence may not fit under a tight budget; consolidation compresses and combines records, improving coverage per token but risking the loss of query-critical details. Neither strategy is universally preferable. This raises two central questions: when should consolidation replace retention, and which operator -- Merge, Abstract, or Rewrite -- should be selected? We formalize this decision by decomposing each operator's utility into a coverage effect on evidence omitted by retention and a signed replacement effect on raw evidence that already fits. Their balance explains why the preferred action changes with relative budget pressure. We implement this mechanism with Offline Abstraction-Safety (OAS), a lightweight learner that estimates action utilities from pre-generation features with held-out harm calibration. The public LongMemEval and LoCoMo benchmarks show the same budget-dependent pattern. On LongMemEval, consolidation improves absolute accuracy by up to 48% under tight budgets, whereas retention is preferable under loose budgets; LoCoMo replicates this crossover at a smaller budget, consistent with its shorter evidence. On both datasets, cross-note abstraction and merging generally outperform local rewriting when compression is necessary.