Keep It InMind: Benchmarking the Implicit-Association Blind Spot in Agent Memory
2026-07-27 • Computation and Language
Computation and Language
AI summaryⓘ
The authors study how long-term memory systems in AI retrieve information based on how similar a user's query is to stored facts. They find a problem, called the implicit-association blind spot, where relevant knowledge is missed because it doesn't look similar to the query (like knowing a macaron contains almonds related to allergies). To explore this, they created InMind, a benchmark with many tasks designed to separate reasons why a fact is missed. Their experiments show that current systems struggle to retrieve relevant facts when the connection to the query is indirect, and the issue lies in how the system decides which facts to bring up, not in the memory itself.
long-term memoryinformation retrievalimplicit-association blind spotInMind benchmarkvector memorygraph memoryembedding dimensionalityquery-conditioned interfacerouting in memory systems
Authors
Ruizhe Li, Mingxuan Du, Benfeng Xu, Zhendong Mao
Abstract
Long-term memory systems store what a user says in an external store and retrieve it when a related query arrives. This interface rests on an assumption so natural that it is rarely stated: a memory that is needed will resemble the query that needs it. World knowledge breaks the assumption. A tree-nut allergy should change the answer to a macaron request through their almond-flour ingredient, yet the two texts share no cue a retriever can see. We call this failure mode the implicit-association blind spot and introduce InMind, a 125-task, expert-verified benchmark spanning ten life domains, with 113 tasks grounded in citable public sources. Its paired controls separate three explanations that existing evaluations conflate: the fact was never stored, the model lacks the bridging knowledge, or the fact was stored and never surfaced. The verdict is clean. With the decisive memory placed in context, the backbone answers 84.0 percent of indirect queries; when the same memory must be retrieved, six vector, graph, and agentic memory systems reach at most 14.4 percent, even though they recall the same facts on demand at up to 100 percent. An embedding with eight times the dimensionality raises answer-blind target recall for every system yet leaves the gap essentially intact. A minimal diagnostic probe that keeps memory visible before the query arrives recovers most of the gap, locating the failure in the query-conditioned interface itself and pointing to routing, deciding which facts must stay visible, as the open problem InMind is built to score.