Memory retrieval improves by using neighborhood-aware residuals

NEAR: Neighborhood-Aware Residuals for Memory Retrieval

Information Retrieval

Summary

Memory retrieval systems search stored records to find answers to questions, but sometimes similar records with different details confuse the search. The authors propose a method called NEAR that looks at related records nearby to adjust how queries and records are compared. This adjustment helps the system better recognize the right evidence by refining the data representations. Tests show NEAR consistently improves memory retrieval and question answering without needing extra training.

What this means in practice

Authors

Han Luo, Zhan Zhuang, Xu He, Jujia Zhao, Yu Li, Suqin Yuan, Dong Li, Lei Feng

Abstract

Memory retrieval aims to identify evidence from stored records to answer a query. Stored records with different factual details may share topics, entities, or events. Such shared contents would make records without the required evidence more likely to receive high similarity scores and outrank the correct evidence. In this paper, we use the information shared by these records to construct local references for query-memory matching. We propose NEighborhood-Aware Residuals (NEAR), a training-free framework that uses these references to construct residual representations that complement the original embeddings. Specifically, each memory derives a reference direction from its neighbors, and each query obtains its reference from a precomputed table. Using these references, NEAR constructs projection residuals for both queries and memories. These residuals are combined with the original embeddings for a single approximate nearest-neighbor search over a fixed memory index. Experiments on eight long-context retrieval benchmarks show consistent improvements across nine embedding models. Downstream evaluations show that NEAR improves question-answering performance.