Large language models show political bias when translating recipes
We're Cooked! - Probing LLM Political Alignment Via Conflict-Framed Recipe Translation
Computation and LanguageArtificial Intelligence
Summary
Large language models (LLMs) are often used to translate text, but this study shows that even small political hints can influence their translations. The researchers found that different LLMs from around the world respond differently when words with conflict meanings like 'enemy' or 'coloniser' appear in recipe translations. Some models avoid taking a side, some silently pick one, and others combine clear reasoning with compliance. This means that these AI models might make hidden political judgments when translating, without telling users.
What this means in practice
- •For translation software developers: Detect and mitigate implicit political biases in AI translation tools, especially when handling sensitive culturally framed content.
- •For content moderation teams: Improve review processes by understanding how subtle language framing can influence AI-generated translations in sensitive or conflict-related contexts.
Authors
Svetlana Gorovaia, Angelica Henestrosa, Ivan P. Yamshchikov
Abstract
Large language models (LLMs) are increasingly deployed for translation tasks, yet their implicit political positioning in such contexts remains understudied. We ask whether a single politically charged framing term, such as aggressor, enemy, neighbour, or coloniser is sufficient to trigger implicit political alignment in an otherwise apolitical task. We present a fully crossed factorial study in which eight models spanning Western, Chinese, and European origins are prompted to translate culturally attributed recipes into a target language left deliberately unspecified. Across 17 languages, four framing conditions, eight models, and 15,680 responses, we find that models do not simply decline or ask for clarification but resolve the ambiguity. Language resolution and reasoning behavior cluster meaningfully along model families: Western models hedge and deflect with vague justifications, Chinese models resolve conflicts silently, and Mistral Large emerges as a distinct profile combining high compliance with conflict-grounded reasoning. Sensitivity to framing terms is consistent across models: even subtle framing variation is sufficient to modulate behavior. Our findings urge caution when deploying LLMs for translation in conflict-adjacent contexts, where implicit political judgments may be made without any signal to the user.