MemDuet speeds up long document reasoning with two-part memory system
MemDuet: Learning Dual-Process Memory for Long-Context Reasoning
Computation and Language
Summary
When computers try to understand very long documents, they often face a trade-off between being accurate and working efficiently. The authors propose MemDuet, a system inspired by how humans think fast and slow. It uses a small model to quickly pick out important information and a larger model to carefully verify and update memory. Their approach is faster and uses fewer steps than previous systems while maintaining good accuracy on long text tasks.
What this means in practice
- •For natural language processing developers: Improve efficiency and accuracy in systems that process very long documents by integrating dual-stage memory mechanisms.
- •For legal document analysis teams: Speed up the review of long legal texts by combining fast evidence retrieval with slower, precise verification for complex reasoning tasks.
Authors
Jiangcheng Song, Haodong Jing, Yulin Mu, Lesong Tao, Jixin Wang, Jin Cui, Tianjing Ren, Runxuan Xie, Yongqiang Ma, Nanning Zheng
Abstract
Reasoning over infinitely long documents is becoming a reality with recurrent memory agents. However, increasing context length exacerbates the trade-off between reasoning performance and inference efficiency. Inspired by Kahneman's account of fast and slow thinking, we propose MemDuet, a dual-process memory framework that learns to coordinate a small language model for fast, parallel evidence selection from raw context chunks (System 1) with a large language model for memory-update verification and memory updates in document order (System 2). We train MemDuet in three stages: supervised fine-tuning of System 1 for evidence selection, Group Relative Policy Optimization (GRPO) of System 2 for verification and memory rewriting, and joint optimization of both policies to coordinate selection and updating for long-context reasoning. Experiments on diverse long-context reasoning benchmarks (RULER-HQA, LongBench v2, and 2WikiMultiHopQA) demonstrate a favorable accuracy-efficiency trade-off as context length grows. Averaged across evaluation settings on the three datasets, MemDuet achieves an end-to-end inference speedup of 9.74x and an output-token reduction of 73.91% relative to RL-MemAgent.