Language models learn to find missing clues for long memory answers
Learning to Retrieve Missing Evidence for Long-Term Memory QA
Computation and LanguageArtificial Intelligence
Summary
Answering questions in long conversations is hard because needed facts might be spread out and the question may not say where to look. The authors created MERA, a method that remembers facts found so far and uses them to find missing information from a big memory store. They taught a small planner using trial and error to ask for info that closes gaps in the evidence. This approach helps big language models give more accurate answers over long chats by gathering scattered clues step by step.
What this means in practice
- •For conversational ai developers: Improve question answering bots to recall and combine scattered information from long dialogues for more accurate responses.
- •For customer support teams: Enhance virtual agents to better handle complex customer queries that require recalling previous conversation details.
Authors
Yi-Xuan Deng, Yi Zhang, Wei Liu, Chao Xue, Shuojin Yang
Abstract
Long-term memory enables language models to use past interactions in future conversations. However, evidence needed to answer a question may be scattered across distant turns, while the question itself omits clues needed to locate it. Retrieved facts can reveal these clues, motivating retrieval decisions conditioned on evidence already found. We introduce MERA (Missing-Evidence Retrieval Augmentation), which separates globally searchable memory from a question-specific evidence state. Verified evidence guides subsequent retrieval without restricting access to the global memory. We train a lightweight planner through reinforcement learning, rewarding queries that recover previously missing evidence. MERA achieves strong answer accuracy across Qwen3-30B and GPT-4o-mini backbones. With Qwen3-30B for evidence processing and answer generation, the trained 0.6B planner achieves 77.40% accuracy on LoCoMo and 71.29% on LongMemEval-S, exceeding a 30B planner without retrieval-grounded training by 4.10% and 3.96%, respectively. On LoCoMo, later retrieval rounds increase cumulative evidence recall from 55.5% to 80.5%.