User identity affects how disclaimers reduce political bias in AI
Whose Guardrail? User Identity Shapes the Effectiveness of Model Disclaimers in Mitigating LLM Political Persuasion
Human-Computer Interaction
Summary
Political bias in AI can influence people's opinions, but warnings about this bias don't always protect users. The authors ran a large experiment showing that disclaimers explaining political bias can help reduce persuasion, but only temporarily and differently depending on the task. People’s political views and how much they use or know about AI also change how much they are influenced. This means simple warnings on AI aren’t enough; we need approaches that consider who the user is and the situation.
What this means in practice
- •For product designers: Design disclaimer messages in AI products tailored to different political views and AI familiarity to more effectively reduce biased persuasion.
- •For platform policy teams: Develop safer AI use policies that recognize diverse user responses to bias warnings instead of relying on generic disclaimers.
Authors
Jillian Fisher, Thomas Richardson, Daniel W. Fisher, Jennifer Pan, Yulia Tsvetkov, Katharina Reinecke
Abstract
Politically biased AI can shift users' opinions, yet it remains unclear whether model disclaimers protect against such persuasion. We report a preregistered experiment (N=1,586) examining how disclaimer design and user characteristics shape susceptibility to politically biased AI. We varied whether disclaimers identified political bias versus potential inaccuracies and whether they included an explanation. Participants completed two decision-making tasks with an LLM biased against their political views, with outcomes assessed immediately and one week later. Disclaimers identifying political bias and including an explanation reduced persuasion, but effects varied across contexts and did not persist. Individual differences were more consistent predictors: Republicans were more persuaded and reported higher trust than Democrats across tasks. Similarly, participants who used AI more frequently were also more persuaded, whereas those with greater self-reported AI knowledge were less persuaded. These findings suggest that generic disclaimers are not enough: effective safeguards must account for user heterogeneity and context.