MemLoc improves long term memory question answering for conversations
Where to Look and What to Use: Retrieve-Localize-Generate for Long-Term Conversational Memory Question Answering
Computation and LanguageInformation Retrieval
Summary
Answering questions from long conversations is hard because relevant information can be scattered across many past talks and some retrieved information is noisy or confusing. The authors propose MemLoc, a method that breaks down past conversations into small units and uses smart retrieval to find the most relevant parts. It then carefully locates the best evidence within those parts to reduce noise and redundancy. Finally, it guides large language models to use this focused information to give accurate answers, improving performance on several tests.
large language modelsretrieval-augmented generationconversational memoryquery routingevidence localizationself-reflective hint policy optimizationcross-memory graphentropy-based granularitylost-in-the-middle effect
Authors
Yifan Wang, Xinkui Lin, Yongxiu Xu, Shen Gao, Ruochen Yang, Kun Huang, Yubin Wang, Jie Wu, Wei Liu, Jian Luan, Hongbo Xu, Shuo Shang
Abstract
Retrieval-augmented generation (RAG) enables large language models (LLMs) to answer questions by accessing external knowledge and has been widely adopted for long-term conversational memory question answering. However, existing methods suffer from two key challenges: (1) fragmented evidence scattered across temporally distant sessions, and (2) noisy content within retrieved sessions that triggers the lost-in-the-middle effect. To address these challenges, we propose MemLoc, a unified Retrieve-Localize-Generate framework for long-term conversational memory QA. For retrieval, MemLoc decomposes each session into multi-granularity memory units and performs query routing via an inner-memory graph with entropy-based granularity selection. It further models cross-session semantic and temporal dependencies through a cross-memory graph, enabling coarse-to-fine retrieval of top-K relevant memory candidates. For localization, we introduce a reasoning-based evidence locator trained with Self-reflective Hint Policy Optimization (SHPO), which performs progressive refinement by extracting query-relevant fragments within memory units to suppress noise and reranking across candidates to remove redundancy, producing a compact evidence set with lightweight location IDs. For generation, these IDs act as precise grounding signals that guide the LLM to the correct memory positions, mitigating the lost-in-the-middle effect while preserving original contextual integrity. Extensive experiments on four benchmarks demonstrate that MemLoc achieves state-of-the-art retrieval accuracy and response quality while maintaining efficiency. Our code is available at: https://github.com/Nikol-coder/MemLoc.