Fact recall errors linked to memory limits in AI models
The Cost of Compression: A Rate-Distortion Limit on Factual Hallucination
Computation and Language
Summary
Sometimes AI gets facts wrong because it doesn’t remember them precisely, not just because it never learned them. The authors show that when a model has limited memory to store facts, it must compress information and this compression can cause errors. They prove a mathematical limit on how well a model can recall facts after compressing them. Their work helps explain one reason why AI sometimes hallucinates false information even when it has seen the truth before.
What this means in practice
- •For language model engineers: Design AI systems with realistic memory limits and understand trade-offs between fact compression and accuracy in closed-book QA.
- •For ai system architects: Optimize memory allocation and retrieval mechanisms in QA systems to reduce hallucinations caused by lossy fact storage.
A theory result. No direct application yet.
Authors
Xi Wang, Shijia Xu, Rongfeng Guo
Abstract
Factual hallucination in closed-book question answering is often treated as a coverage problem: a model fails because the relevant fact is absent from its internal memory. This view misses a second source of error. Even when a fact has been observed, finite memory may force it to be stored only approximately. We study this effect through a simple coverage--compression model of factual recall. We consider an unstructured question-answering task with $N$ possible queries and $K$ possible answers. A learner observes $M$ training facts, compresses them into at most $B$ bits, and answers uniformly drawn test queries without retrieval. For a uniformly random ground-truth mapping, we prove $\mathcal{E} \geq \frac{M}{N}δ^\star\!\left(\frac{B}{M}\right) + \left(1-\frac{M}{N}\right)\left(1-\frac{1}{K}\right)$, where $δ^\star(r)$ is the inverse rate-distortion function of a uniform $K$-ary source under zero-one loss. The two terms separate compression distortion on observed facts from missing coverage on unobserved facts. The bound gives a compact way to reason about selective memory, forced compression, structure, retrieval, abstention, and long-context organization. We study the predicted signatures with theory-implied simulations and controlled fact-injection probes in modern language models that vary fact load and effective trainable memory. The result is not a complete theory of hallucination, but an information-theoretic account of a separable failure mode: lossy recall of observed facts under finite memory.