Longzero improves large language models reasoning on long complex texts
LongZero: Enhancing Long-Context Reasoning in LLMs from Zero Data
Computation and LanguageMachine Learning
Summary
Large language models often struggle to think through long, complicated information because they need lots of tricky examples to learn from. The authors present LongZero, a way for these models to create their own practice problems without any outside data. LongZero helps models break down big problems into smaller steps and then build up full stories, improving their ability to reason over long texts. Tests show this method helps models perform better on tasks that need understanding of long documents.
What this means in practice
- •For natural language processing engineers: Train language models to handle longer and more complex reasoning tasks without requiring costly annotated data sets.
- •For enterprise ai developers: Build AI tools that better understand and process long multi-document contexts for customer support or knowledge management.
Authors
Zhennan Wan, Jianfei Chen
Abstract
Reinforcement learning (RL) is an effective way to strengthen the long-context reasoning of large language models (LLMs), but it needs training data that are long, diverse, challenging, and verifiable. Human annotation of such data is expensive, naturally occurring data are too scarce to sustain RL, and existing synthesis methods depend on external corpora or stronger teacher models, which become harder to obtain as the trained models get stronger. Recently, RL from zero data, in which a model learns from tasks it proposes itself, has achieved excellent performance, but existing methods target short-context tasks. Therefore, we propose $\textbf{LongZero}$, a method that lets an LLM improve its own long-context reasoning without any external data. Instead of writing questions and answers over existing documents, LongZero first has the model synthesize the question and answer with its supporting evidence in a compact structured form, and only then separately expands the evidence into a long multi-document context. LongZero combines these self-written multi-hop reasoning tasks with self-written multi-round retrieval tasks and trains the model with GRPO. We evaluate LongZero on two LLMs across five long-context reasoning benchmarks and compare it with RL on data synthesized from external corpora. Experimental results show that LongZero can generalize to real tasks, and outperforms both baselines, suggesting a viable and promising path to improving long-context reasoning of LLMs.