Context language models improve memory and compute efficiency in AI tasks
Context Language Models
Artificial IntelligenceComputation and LanguageMachine Learning
Summary
Language models usually rely on external programs to keep track of information they need while working. The authors created a new type of model that manages its own memory by treating context like a file it can change freely. This means the model learns what information to keep and update, working better and faster in multi-agent setups too. Their experiments show these models perform better and use less computing power than previous methods, and they can improve over time using natural language instructions and reinforcement learning.
What this means in practice
- •For ai platform engineers: Enable AI services to manage memory internally for better performance and lower compute costs across various tasks and workloads.
- •For cloud infrastructure teams: Reduce server-side compute usage by adopting suffix cache reuse techniques designed for context language models.
- •For automated customer support developers: Build multi-agent AI systems that efficiently maintain and update shared conversation context without external orchestration.$Commercial implications: This enables creation of scalable, self-managing AI support agents that can improve customer interaction quality with reduced operational costs.
Authors
Rulin Shao, Shannon Zejiang Shen, Junjie Oscar Yin, Yuetai Li, Minheng Wang, Hamish Ivison, Radha Poovendran, Nathan Lambert, Teng Xiao, Mike Lewis, Wen-tau Yih, Luke Zettlemoyer, Pang Wei Koh
Abstract
We introduce Context Language Models (CLMs), language models that natively manage their own context. We implement this by treating the context as a file and allowing the model to make unrestricted updates to this file. This allows the model to learn what is most important to maintain in context, and naturally extends to multi-agent systems where multiple agent contexts coexist as files. Building CLMs zero-shot with existing models outperforms SOTA context management strategies across a variety of tasks: 11.4% higher accuracy with 21.5% fewer FLOPs on BrowseComp-Plus, 5% higher scores with 59% fewer FLOPs on 12-hour EdgeBench, and 65% greater improvement with the same compute on a 24-hour multi-repository agent-swarm task. Moreover, by shifting context management from external harness control to intrinsic model behavior, CLMs naturally enable both in-context and parametric learning of context-management strategies. We show that CLMs can be steered with natural-language instructions evolved through a standard skill-optimization loop, improving held-out accuracy by up to 35.9 points on a context-management task while reducing compute. We also introduce an online reinforcement learning method for CLMs, improving Qwen3.5-9B performance on BrowseComp-Plus by 47.6% while using 12% fewer FLOPs. Finally, we co-design Suffix Cache Reuse for CLM serving, further reducing server-side compute by 35% relative to standard SGLang at matched performance.