AI language models distort the stance in human communication
Stance Drift: How AI-mediated Communication Distorts Our Message
Computation and Language
Summary
The authors studied how large language models (LLMs) change the original opinion when they rewrite and interpret messages. They found that these AI systems often shift the stance away from what was originally intended, sometimes making it more extreme or even flipping it. Testing various LLMs, none perfectly preserved the original stance, though some reasoning techniques improved accuracy a bit. This matters because miscommunication via AI tools could affect important areas like journalism and policy discussions.
What this means in practice
- •For journalists: Evaluate and mitigate stance distortion when using AI tools for drafting or summarizing opinionated content.
- •For policy analysts: Assess the risks of altered stances in AI-mediated communication during public consultation and policy review processes.
Authors
Lingchong Liu, Yanfei Zhou, Jacob Bien, Y. X. Rachel Wang, Lucy Xia, Xin Tong
Abstract
Large language models (LLMs) increasingly mediate human communication, from drafting emails to summarizing scientific reports, yet whether they faithfully preserve a speaker's position remains largely untested. We model AI-mediated communication as a two-step generation-extraction pipeline: one LLM produces an argument from a specified stance, and a second LLM extracts the stance from that argument. We represent the pipeline as a probabilistic state transition over five Likert-type stance categories and define the stance preservation rate (SPR) as the average probability that the extracted stance matches the initial stance. Across 112 debate propositions, none of the nine LLMs tested exceeded an SPR of 0.7 under the default configuration. Three drift patterns accounted for most of the drift: polarization, deviation from neutrality, and flipping. Among the mitigation strategies tested, including in-context learning, multiple extraction with shuffled options, assertion, and reflection, only adding medium reasoning effort to a reflection prompt for GPT-5.4 substantially improved the SPR, to 0.775, yet polarization remained the largest pattern, with 0.119 of the transition mass. An exploratory comparison with human extraction on a single proposition suggests that drift arises at both the generation and the extraction stage. These results point to a fidelity gap in AI-mediated communication, with implications for journalism, policy deliberation, scientific communication, and other domains where opinion-laden messages pass through language models.