ConvMem method speeds up long text reasoning with parallel processing
ConvMem: Convolutional Memory for Long-Context Reasoning
Artificial IntelligenceComputation and Language
Summary
Large language models find it hard to understand very long texts because they have fixed memory limits. The authors introduce ConvMem, a way to process long passages faster by treating the model’s work like a layered puzzle solved in parallel steps. This method avoids the slow, step-by-step reading used before and doesn’t need extra training that can cause mistakes on new topics. Tests show ConvMem does better than other methods that don’t require training and is more reliable on tasks it wasn’t specifically trained for.
What this means in practice
- •For software developers: Enable faster and more efficient processing of very long documents within language model applications by using ConvMem’s hierarchical and parallel approach.
- •For legal tech teams: Process and reason over extensive legal texts and case documents more efficiently without costly retraining or degraded accuracy on unseen cases.
Authors
Hongming Zhang, Zhaozhen Gu, Fengshuo Bai, Ming Hao, Qingyang Zhang, Yuanyuan Wang, Shiyang Tang, Yanna Wang, Bo Xu
Abstract
While Large Language Models (LLMs) have demonstrated impressive capabilities, they often struggle with extremely long contexts due to fixed context limits. To address this, sequential approaches like MemAgent extend the effective context by reading text in segments and iteratively updating a fixed-size memory. However, this sequential paradigm suffers from high latency and requires costly reinforcement learning (RL) training, which can lead to overfitting on specific datasets. To overcome these limitations, we propose ConvMem, a training-free, highly parallelizable framework that reformulates long-context reasoning as a hierarchical convolution. Inspired by CNNs, ConvMem treats an LLM prompted with a specific query as a convolutional kernel. This kernel summarizes text segments hierarchically, shortening the reasoning path from a linear chain into a logarithmic tree. Specifically, ConvMem integrates \textit{Configurable Strides} and \textit{Skip Connections} to ensure robust evidence capture and propagation, while employing \textit{Multi-Kernel Convolution} to decompose complex queries into disentangled semantic channels. This design not only mitigates error accumulation but also enables massive parallelization across both text segments and reasoning threads. Experiments on RULER-HotpotQA and RULER-2WikiMultiHopQA demonstrate that ConvMem outperforms training-free baselines and avoids the risk of overfitting to parametric priors often observed in RL-trained models on out-of-distribution tasks.