Causal memory policy improves memory usefulness estimation in large language models
Causal Memory Policy: Making Memory Utility Identifiable by Intervening on Retrieval
Artificial Intelligence
Summary
Large language models that use memory need to decide which past memories to keep for future tasks. The authors found that many memories never get evaluated properly because they are never retrieved, so their usefulness cannot be measured. To fix this, they developed a method called Causal Memory Policy (CMP) that deliberately retrieves memories with known probabilities, allowing a better estimate of how useful each memory really is. Their method improves identifying which memories matter but also shows that knowing a memory's usefulness for one question doesn’t guarantee it will be useful for others.
What this means in practice
- •For ai system engineers: Improve memory retention decisions in large language models by accurately estimating the utility of individual memories through controlled retrieval interventions.
- •For recommendation system developers: Use causal intervention methods on retrieval operations to better identify and weigh relevant past data items that affect user outcomes.
Authors
Arman Behnam, Binghui Wang
Abstract
Memory-augmented large language models must decide which memories to retain, and recent systems do so by estimating each memory's effect on task performance. However, these estimates rely entirely on retrieved memories. When a memory is never retrieved, store-level interventions produce identical outcomes, leaving its utility unidentified. This is a retrieval-level positivity violation, invisible to diagnostics that examine only memory operations. We introduce Causal Memory Policy (CMP), a causal framework that restores identification by intervening on retrieval itself, reserving a fixed number of context slots for memories sampled with known propensities. CMP estimates memory utility by self-normalized inverse propensity weighting under a balanced assignment design. We prove the causal factorization of memory utility through retrieval, the unbiasedness and exact variance of the estimator, and the optimal decision rule under irreversible operations. Empirically, identification fails for 54% of required memories on LongMemEval and 67% on LoCoMo, and the failure persists in a deployed memory system. CMP improves discrimination between required and non-required memories from 0.54 to 0.66 AUC. Finally, we show that identified memory utility alone is insufficient for retention decisions: per-query utility reaches 0.78 AUC on the query for which it is estimated, yet no aggregation available to a retention policy predicts a memory's value on unseen queries. Code is available at: https://anonymous.4open.science/r/cmp-release-D0C3/.