Memory system stores full documents for better long-term answers
Mem++: Non-Destructive Memory for Long-Term Organizational LLM Agents
Computation and LanguageArtificial Intelligence
Summary
When groups use language AI to keep track of decisions over time, it's hard to remember which version is correct at any moment because older notes get summarized or changed. The authors created Mem++, a system that saves every document exactly as it was written, with dates and authors, without altering them. When asked a question, Mem++ looks up all relevant documents up to that date and uses a mix of word matching and meaning to find the best answer. This approach keeps all versions safe and improves how well the AI can answer questions about past decisions.
What this means in practice
- •For enterprise knowledge managers: Maintain complete histories of organizational decisions for accurate retrieval without losing past versions.
- •For customer support teams: Access all prior versions of help documents to find contextually relevant answers to customer questions over time.
Authors
Ahmad Yehia, Aly O. Abdelkareem, Islam Ahmed, Hesham Omran, Khaled Alashmouny, Christian Claudel, Abduallah Mohamed
Abstract
Large Language Model (LLM) agents now take part in organizational work, where many authors record decisions across documents over months. Because a revised decision arrives as a new document rather than an edit, answering a question requires knowing which version held at a given time. However, most memory systems compress the record at write time. By distilling each document into facts, notes or graph edges, these methods fix what can be answered before any question is asked. To address this, we propose Mem++, a non-destructive memory framework shifting from write-time distillation to read-time selection. Mem++ stores every document whole with its date and author, and it calls no generative model at write time. At read time, it retrieves only documents dated up to the time a question asks about and fuses lexical and semantic rankings. Unlike systems that overwrite older versions, Mem++ keeps them and leaves the choice to the answering model. Evaluations on the organizational benchmark OrgMemBench demonstrate that Mem++ surpasses the strongest memory system baseline by 8.0 to 13.1 points across two answering models. With gpt-4.1-mini, it also achieves the best overall score, 2.6 points above RAG. In addition, Mem++ achieves the best average LLM-judge score on LoCoMo and ranks second on LongMemEval-S, behind only its entity-graph variant. Code for benchmark evaluation is available at https://github.com/AIDAChip-Inc/mem-plus-plus.