Summary
Language models often use fixed context files loaded at the start of a session to guide their behavior. The authors study how the size and content of these files affect performance since every token is counted repeatedly and can drag down efficiency if the file grows too large. They show that simply adding every useful instruction is not always best and that selecting an optimal subset improves overall results. They also explore how feedback from past interactions, which is incomplete or 'censored,' can guide which instructions to keep, add, or remove. Their work helps understand how to maintain these context files so language models work better over time.
What this means in practice
- •For ai platform engineers: Optimize which instructions to keep in fixed context files to improve language model performance over many sessions while respecting token limits.
- •For human-in-the-loop system developers: Design feedback and update processes that handle incomplete feedback to refine language model context files effectively under limited reviewer capacity.
Tested on simulated data.
Abstract
At the start of every session, LLM agents load a fixed context file, such as $\texttt{AGENTS.md}$. Each loaded token in the file is charged again in every later round of the session, and these files can degrade performance as they grow in size. However, in practice, human or automated curators usually grow these files by appending. We formulate context curation as a capacitated assortment problem. Instructions consume tokens under a finite attention capacity; adding an instruction never raises the compliance of the others, while retained instructions incur a per-session setup cost. We prove an upper bound on the optimal file size, regardless of the number of available candidate instructions, and that appending every instruction with positive standalone value can be arbitrarily worse in net value than selecting an optimal subset. A token budget also limits the loss when the token price is underestimated. We then examine what can be learned from past sessions and how this information can guide decisions to add or remove instructions. Feedback is inherently censored: the benefits and harms of loaded instructions are observable, whereas missing instructions generate feedback only when their absence causes harm. In this setting, we show that deleting instructions ignored by agents can inevitably remove helpful ones. We characterize how much evidence should be collected before adding an instruction. Besides, we bound regret when human reviewers can inspect only a limited number of edits per period. Empirical experiments further show that irrelevant rules drawn from real context files reduce language-model compliance.