Memory decision controller cuts hallucinations in language models
An Interpretable Memory Decision Controller for LLM Agents Based on Three-Signal Complementarity: Decoupling Confidence and Consistency
Computation and Language
Summary
Large language models often retrieve information from memory to help answer questions, but when the stored memories conflict, the models can get confused and make up wrong answers. The authors created a Memory Decision Layer (MDL), a simple and fast step that helps the model decide which memories to trust based on three signals: how relevant, reliable, and risky the memory is for the task. This method reduces the wrong answers caused by confusing memories by over half and almost eliminates errors in tricky situations, without needing to train new parts of the model.
What this means in practice
- •For language model engineers: Improve language model outputs by adding a fast memory filter that reduces hallucinations when injected memories conflict.
- •For automated customer support teams: Enhance chatbots’ answer reliability by integrating a decision layer that signals when to trust or reject retrieved knowledge.
Authors
Yiming Zhang, Jinghong Zhang, Haoran Zhao, Yiren Ma, Chunlei Zhao
Abstract
Memory systems for large language models have focused predominantly on efficient retrieval, whereas the decision of whether retrieved memories should be trusted has received comparatively little attention. When the memory store contains conflicting positions, standard retrieval-augmented generation (RAG) blindly injects memories and amplifies hallucinations: in models susceptible to memory injection, the RAG hallucination rate under conflicting memories is markedly higher than that of a memory-free baseline. Inspired by memory signaling mechanisms in the prefrontal cortex, we propose the Memory Decision Layer (MDL), a zero-parameter memory decision controller situated between the retrieval and generation stages. Its core is a three-signal complementary encoder that fuses relevance, reliability, and task risk through QR-based orthogonal subspace projection and a meta-working-memory signal into an interpretable decision representation that quantifies the trustworthiness of retrieved memories. Building on this encoder, MDL explicitly decouples confidence from consistency and introduces risk inversion and explicit abstention. Evaluations on mainstream large language models and multiple open-source datasets show that MDL reduces the hallucination rate under conflicting memories by about 56.04% in general scenarios and approaches zero hallucination in high-risk scenarios. The controller is fully white-box: it relies purely on geometric operations, requires no trained parameters, and adds only about 0.14 ms per decision -- roughly 50x faster than the embedding-retrieval step that precedes it and four to five orders of magnitude faster than an LLM self-evaluation call.