Memory lifecycle management matters for better language agent memory use
VitaBench: Memory Lifecycles Beyond Retrieval
Machine Learning
Summary
Memory systems in language agents not only have to find old information but also decide how to keep it—exact, compressed, or deleted—before knowing if it will be needed again. The authors introduce VitaBench, a tool that tests how well these memory decisions are made over time by replaying memories and seeing long-term effects. They found that making smart choices about memory storage is different from just retrieving information and can improve performance when delayed outcomes are considered. Their tests show many situations change the best memory action depending on how far into the future you look.
What this means in practice
- •For chatbot developers: Evaluate and improve memory management policies that decide which user interactions to keep or discard over long conversations.
- •For customer support platform teams: Use realistic interaction streams to test AI memory adaptation for better service continuity and information retention.
Authors
Muquan Yu, Dihan Hu
Abstract
Long-term memory benchmarks for language agents primarily evaluate whether information from earlier interactions can be retrieved and used when relevant. Persistent memory systems face an additional problem before future use is known: deciding whether a memory should remain exact, be compressed, consolidated, or deleted. Existing benchmarks capture long-range retrieval, temporal reasoning, evidence reuse, and incremental interaction, but do not directly evaluate these earlier representation decisions. We formalize memory lifecycle evaluation and introduce VitaBench, a benchmark that makes lifecycle decisions measurable through repeated memory reviews and full-horizon counterfactual evaluation. VitaBench contains a controlled Diagnostic tier that isolates future-use patterns and a Naturalistic tier that composes them into generated customer-service-style interaction streams. We characterize lifecycle decision structure with three diagnostics: lifecycle headroom, measuring the advantage of adaptive choices over the best fixed strategy; horizon reversal, measuring when short-horizon decisions change under delayed consequences; and optimal-action diversity, measuring whether different memory states favor different lifecycle operations. Across both tiers, VitaBench exhibits substantial lifecycle structure: lifecycle headroom is 1.364 in Diagnostic and 1.405 in Naturalistic, and 60.2% and 56.5% of states, respectively, change their preferred action between short and full horizons. A learned delayed lifecycle policy further improves over the best fixed strategy on VitaBench-Naturalistic, reaching 101.5 versus 83.4 joint utility. These results show that retrieving past information and deciding how it should persist are distinct evaluation problems, and that lifecycle management requires evaluation of the delayed consequences of representation decisions.