Distributed approach improves reasoning on very long AI texts
DISCO: Distributed Long Context Scaling with Grounding-Reasoning Disaggregation
Computation and Language
Summary
Large language models often struggle to think clearly when given very long texts, a problem known as context rot. The authors propose DISCO, a system that breaks down these long texts into parts handled in parallel by multiple smaller AI units focused only on finding specific information. A central AI plans and combines these results to answer questions better. This method helps maintain accurate reasoning on texts that are millions of words long, while using much less computing power.
What this means in practice
- •For enterprise ai teams: Enable AI systems to accurately process and reason over documents with millions of words without loss of performance or excessive costs.
- •For cloud service providers: Deploy more efficient AI inference services that handle very long text inputs with significantly lower computing resources and costs.$Commercial implications: This paper enables offering cost-effective long text AI processing as a service to enterprise customers.
Authors
Guanzheng Chen, Viet Dac Lai, Subhojyoti Mukherjee, Branislav Kveton, Seunghyun Yoon, Franck Dernoncourt, Qizhe Xie, Trung Bui
Abstract
While Large Language Models (LLMs) advertise million-token context windows, reasoning quality often collapses as inputs grow -- a phenomenon termed context rot. This failure stems from a structural entanglement in monolithic architectures, where the massive search burden of contextual grounding exhausts the representational capacity needed for complex reasoning. To resolve this, we propose Grounding-Reasoning Disaggregation via DIStributed long COntext scaling (DISCO). Inspired by distributed computing frameworks like Apache Spark, DISCO partitions long context across a fleet of Worker LLMs dedicated exclusively to parallel, localized grounding. A central Driver LLM, trained via Reinforcement Learning (GRPO) to optimize planning, orchestrates execution by dynamically mapping queries into atomic extraction tasks and reducing the gathered evidence to synthesize a final answer. By isolating reasoning from raw context noise, DISCO effectively eliminates context rot. On RULER-QA (1M tokens), it maintains 78.4% accuracy where standard baselines collapse. Furthermore, it outperforms full-context models by up to 9.8 points on LongBench v2 and matches frontier models like Gemini-3-Pro-Preview while reducing inference costs by over 80%, establishing a highly efficient paradigm for robust long-context inference.