AgentMemGate stops false memories in conversational AI assistants
AgentMemGate: Addressing Speculation Contamination in Conversational Assistant Memory
Artificial Intelligence
Summary
Conversational AI assistants often remember things users say, but sometimes they record plans or guesses as if they are facts, which can cause confusion later. The authors call this problem speculation contamination. They created AgentMemGate, a system that decides when a statement is just a plan or speculation and stops it from being incorrectly remembered as fact. Their method improved accuracy a lot and reduced errors in remembering user information during conversations.
What this means in practice
- •For conversational ai developers: Improve memory accuracy of AI assistants by preventing speculative user statements from being stored as facts.
- •For customer support platforms: Enhance multi-turn chatbot memory to handle user plans or uncertain information without causing incorrect responses.
Authors
Chirag Sharma, Benjamin Fowlersmith, Karime Maamari
Abstract
Conversational AI assistants with long-term memory extract facts from user messages into a store consulted in later conversations. A stated plan can enter that store as fact: a user who might move to Seattle may be recorded as already living there. We call this speculation contamination. Final-state memory benchmarks miss this error because they do not probe intermediate state and include few unresolved speculations. We present AgentMemGate, a write-time gate for profile-store memory that classifies extracted statements as speculation, completed event, correction, or other. Speculations remain outside memory, with conditions governing later promotion or deletion. We also contribute a dataset of multi-session conversations in which plans are confirmed, abandoned, or left unresolved. On our 147-conversation held-out set, Mem0 and Graphiti assert unresolved plans as current state for 35.2% and 27.3% of pending plans. On the core benchmark, AgentMemGate eliminates all observed contamination relative to the identical ungated pipeline (87.5% to zero for the most exposed extraction style) and raises task accuracy from 65% to 95%. On the harder held-out set, gated contamination is 3.4% to 5.7% and task accuracy rises by 9 to 13 percentage points. Our analysis identifies field matching as the main remaining bottleneck: realistic speculations often match no profile field and never reach the gate. We release our datasets, prompts, and evaluation code.