It's Not What You Say, It's How You Say It: Evaluating LLM Responses to Expressions of Belief
2026-07-20 • Computation and Language
Computation and Language
AI summaryⓘ
The authors studied how different ways people express beliefs when talking to large language models (LLMs) affect whether the models accept those beliefs or rely on their own knowledge. They created a system categorizing expressions of belief by form, evidence, certainty, and tone, with 17 detailed types. Testing 16 different LLMs, they found that larger and instruction-trained models are less likely to accept user beliefs compared to smaller or base models. Some types of expressions were more effective at convincing the models than others. This work helps understand how language style influences LLM behavior, which is useful for improving prompts and model reliability.
Large Language ModelsExpressions of BeliefEvidentialityEpistemic StanceTonePrompt EngineeringModel RobustnessContext Integration
Authors
Kevin Du, Clara Kümpel, Michelle Wastl, Alex Warstadt
Abstract
Users frequently express their beliefs to large language models (LLMs). In some situations, the LLM should accept these contextual beliefs as true. In others, they should stick to their prior knowledge. Notably, users' expressions of belief (EoBs) can take linguistically diverse forms - using presuppositions, evidential and certainty markers, or varied tones - each of which may have a different persuasiveness over the LLMs. We introduce a typology to systematically evaluate how different EoBs affect whether models follow context versus prior knowledge. The typology is grounded in four linguistically motivated dimensions: form, evidentiality, epistemic stance, and tone, spanning 17 fine-grained types. By pairing these EoBs with world knowledge facts, we generate controlled EoB-query pairs that isolate the effect of linguistic variation. Using this benchmark, we evaluate 16 LLMs that differ in architecture (Llama3, Qwen3, Gemma3), scale (1B-30B parameters), and training stages (base vs instruct). We identify meaningful variations in response behavior across these axes, e.g., that bigger models and instruction models tend to be less context-following than smaller models and base models. We further identify specific EoBs that statistically significantly persuade LMs more consistently than others. Our work reveals systematic patterns in how linguistic framing affects LLM context integration, with implications for prompt engineering and model robustness.