Diagnosing and Mitigating Context Rot in Long-horizon Search
2026-06-29 • Information Retrieval
Information RetrievalArtificial IntelligenceComputation and Language
AI summaryⓘ
The authors study a problem called 'context rot,' where large language models (LLMs) perform worse as they get more information to process, especially in tasks that need long context. They found that when given too much context, models either stop trying or give uncertain answers, and this gets worse as the context grows. By experimenting with different models and methods, the authors showed how managing the context and using special filtering techniques can reduce this problem and improve model performance. They also found that combining these approaches works even better.
Large Language Modelscontext lengthcontext rotdeep searchcontext managementrejection samplingpruning experimentsopen-source modelsaggregation methods
Authors
Shijie Xia, Yikun Wang, Zhen Huang, Pengfei Liu
Abstract
Extensive context has become the norm as Large Language Models (LLMs) are increasingly deployed in long-horizon tasks. The concern that increasing context length degrades model capabilities, known as context rot, has become a central issue for these applications. In this paper, we focus on deep search scenarios, aiming to investigate the rot phenomenon and its mitigation strategies. By evaluating four flagship open-source models across three benchmarks, we reveal a prevalent but unnoticed rot phenomenon: extensive context causes models to directly give up or prematurely provide uncertain answers, and this issue is exacerbated as the context grows. Through pruning experiments, we demonstrate the relationship between the accumulated context and the rot phenomenon. Furthermore, we investigate mitigating this issue through context management and post-hoc rejection sampling. For context management, we systematically evaluate seven different methods across three categories, based on performance, cost, and impact on context rot, providing clear guidance for strategy selection and usage. For rejection sampling, we develop a rot-aware filtering strategy and demonstrate its effectiveness across three aggregation methods. Finally, we show that these two approaches can be combined for further performance improvements.