Discourse-Role Labels as Presentation-Time Variables for Context Use in Language Models
2026-06-02 • Computation and Language
Computation and Language
AI summaryⓘ
The authors studied how adding different labels like Instruction: or Example: to supplied text affects how language models use that information. They tested multiple models with the same misleading content wrapped in various labels and found that some labels made models more likely to accept the wrong answer, while others like Example: reduced this effect. Their analyses show that the label surrounding content influences model responses, suggesting that benchmarks should consider these labels because presentation impacts model behavior. They confirmed these findings through tests and manual checks to ensure reliability.
context-augmented language modelsdiscourse-role labelsMisleading Adoption RateMMLU-ProRAG (retrieval-augmented generation)log-probabilitybenchmarkingmanual auditinstruction tuningmodel behavior
Authors
Jianguo Zhu
Abstract
Context-augmented language model systems often wrap supplied content with labels such as Reference:, Evidence:, Instruction:, Note:, or Example:, but the effect of these labels on reader-model behavior remains underexplored. We introduce a paired fixed-content probe over 500 MMLU-Pro items: each item receives the same misleading answer-bearing assertion under different discourse-role labels, and adoption is measured by whether the model outputs the injected wrong option. Across GPT-5.5, DeepSeek V4 Pro, Llama-3-8B-Instruct, and Qwen2.5-7B-Instruct, Misleading Adoption Rate shifts by 56-84 percentage points. Binding or source-like labels such as Instruction: and Reference: produce high adoption, whereas Example: consistently suppresses it. Paired tests, bootstrap intervals, final-instruction ablations, and Qwen final-step log-probability probes support a label-conditioned candidate preference. Boundary probes show where the effect weakens or persists: arithmetic tasks reduce adoption, passage-shaped external context preserves smaller label gaps, short-answer evaluation rules out option-letter copying, and nested-label conflicts suggest that illustrative framing can delimit adoption scope. A 200-case single-author manual audit confirms that the short-answer contrasts are stable under conservative adjudication. The resulting claim is bounded but practical: context-utilization and reader-side RAG benchmarks should report and control wrapper labels, because presentation choices can change measured reliance on supplied context.