MemReread: Enhancing Agentic Long-Context Reasoning via Memory-Guided Rereading
2026-05-11 • Computation and Language
Computation and LanguageArtificial Intelligence
AI summaryⓘ
The authors address the problem of understanding very long documents without slow or complicated methods. They propose MemReread, a technique that reads through text in order but can pause to break down questions and reread parts if it realizes it missed important details. This method avoids the need to search back through old information repeatedly, making reasoning more efficient while still capturing important facts. They also use reinforcement learning to decide how many times to reread based on how tricky the problem is, keeping the process fast. Tests show MemReread works better than earlier approaches on long reading tasks and stays quick even for very long texts.
long-context reasoningattention mechanismagent memoryretrieval-based recallquestion decompositionreinforcement learninglinear time complexitymemory overwritingstreaming readinglength extrapolation
Authors
Baibei Ji, Xiaoyang Weng, Juntao Li, Zecheng Tang, Yihang Lou, Min Zhang
Abstract
To tackle long-context reasoning tasks without the quadratic complexity of standard attention mechanisms, approaches based on agent memory have emerged, which typically maintain a dynamically updated memory when linearly processing document chunks. To mitigate the potential loss of latent evidence in this memorize-while-reading paradigm, recent works have integrated retrieval modules that allow agents to recall information previously discarded during memory overwriting. However, retrieval-based recall suffers from both evidence loss during memory formation and interference induced by invalid queries. To overcome these limitations, we propose MemReread. Built upon streaming reading, MemReread circumvents intermediate retrieval. It triggers question decomposition and rereading when the final memory is insufficient, enabling the recovery of indirect facts that were prematurely discarded. This design supports non-linear reasoning while preserving the inherent logical flow of document comprehension. To further enhance practicality, we introduce a reinforcement learning framework that enhances length extrapolation capability while dynamically determining the number of rereading passes based on task complexity, thereby flexibly controlling computational overhead. Extensive experiments demonstrate that MemReread consistently outperforms baseline frameworks on long-context reasoning tasks, while maintaining linear time complexity with respect to context length.