Emergence of Context Characteristics Sensitivity in Large Language Models
2026-06-08 • Computation and Language
Computation and LanguageArtificial Intelligence
AI summaryⓘ
The authors studied how large language models learn to use instructions during different training steps. They found that after the first step, models prefer contexts that are easy to understand, such as longer texts and ones similar to the question. Later training steps can strengthen or weaken these preferences based on the data used. Their work shows that how models use context changes throughout training, and choosing the right training data is important to help models use context better.
instruction fine-tuninglarge language modelscontext usagesupervised fine-tuningdirect preference optimizationreinforcement learningcontext-query similarityfluencytraining dataset
Authors
Nadya Yuki Wangsajaya, Haeun Yu, Isabelle Augenstein
Abstract
During instruction fine-tuning (IFT), large language models (LLMs) learn to follow instructions by using the provided context to answer a query. While prior work has studied how context characteristics correlate with context usage by the LLM, this analysis has been limited to inference time, leaving open how these relationships are acquired in the first place. Here, we measure how models' sensitivity to such characteristics shifts across successive IFT stages: supervised fine-tuning (SFT), direct preference optimization (DPO), and reinforcement learning with verifiable rewards (RLVR). Experiments across four models and three datasets show that SFT makes models more likely to use contexts that are easy to understand, such as containing high length, context-query similarity, and fluency. Post-SFT dynamics may either reinforce or resolve these preferences depending on the training dataset. Our findings reveal that context usage is actively reshaped at each IFT stage, and designing a balanced IFT dataset is important in ensuring robust context utilization of instruction-tuned models.