Agent memory systems reveal varied memory update and retention behaviors
Probing Stability-Plasticity Tradeoffs in Agent Memory through Cognitive Experimental Paradigms
Computation and LanguageArtificial IntelligenceMachine Learning
Summary
Many computer programs use memory to remember user preferences and important information over time, but they are usually judged only by how correct their final answers are. The authors created MemProbe, a new way to test how these programs balance keeping old memories stable and learning new ones. Inspired by how human memory works, MemProbe includes experiments that see how programs handle confusing or changing information, whether they keep memories strong, and how they change memories when reactivated. They tested six memory systems and showed that even if they get similar scores overall, their detailed memory behaviors are quite different.
What this means in practice
- •For conversational ai developers: Diagnose how different memory designs impact user preference updates and retention over long interactions.
- •For personalized recommendation teams: Analyze memory behaviors to improve how systems handle evolving user tastes versus preserving known preferences.
Authors
Jiaqi Ding, Guorong Wu
Abstract
Agent memory systems are increasingly used to maintain long-term user preferences, task states and evolving facts, but current evaluations often collapse memory behavior into final-answer accuracy. We introduce MemProbe, a cognitive-science-inspired framework for diagnosing stability-plasticity tradeoffs in agent memory. The framework is motivated by a core insight from cognitive memory research: memory is reconstructive and shaped by interference, source reliability, reinforcement, and reactivation. MemProbe turns this insight into four reusable experimental paradigms (interference, misinformation, consolidation strength, and reconsolidation window) that manipulate when a memory should be updated, preserved, or treated as uncertain. It further decomposes correctness into behavioral profiles that reveal how systems update, preserve, attribute, and temporally organize information. We instantiate these paradigms in a 56-episode diagnostic suite and evaluate six incremental memory systems under a unified protocol. Results show that systems with similar aggregate scores exhibit distinct behavioral profiles. MemProbe provides such a diagnostic lens, turning aggregate performance into interpretable profiles of memory maintenance over time. Code is available at https://github.com/jq-ding/MemProbe.