Language and country cues affect large language model cultural views differently
Investigating Multi-Cultural Value Selection Biases in LLMs
Computation and Language
Summary
Large language models are asked to reflect cultural values, but they often rely on just one cultural label when people have multiple identities. The authors studied how these models respond when given two cultural clues: a country and a language. They found that language cues change the model’s answers more than just naming a country, but not always in a way that matches that country’s typical views. This means that language and country labels should be treated as separate signals in how these models understand culture.
What this means in practice
- •For chatbot developers: Design culturally aware chatbots that distinguish between users’ language and country to better tailor responses.
- •For multilingual app developers: Improve app interactions by separately handling language and country signals for more accurate cultural assumptions.
Authors
Siddhesh Pawar, Arnav Arora, Dustin Wright, Dirk Hovy, Isabelle Augenstein
Abstract
Large language models (LLMs) serve users whose cultural identities are multi-layered, yet cultural evaluations commonly reduce identity to one label. In this work, we study LLMs under simultaneous identity cues, with a country persona in the system prompt and a second identity signaled in the user turn, either by naming a country (explicit cue) or through the query language (implicit cue). We test World Values Survey (WVS) multiple-choice probing and open-ended sociopolitical generation. In multiple-choice probing, language cues shift answers about three times more than explicit cues but away from the associated country's distribution, and explicit cues do not consistently move answers toward the named country. Across 1.6 million open-ended responses, naming a country closes 30-44% of the gap to the model's own responses for that country, and explicit cues produce more differentiated value changes than language cues. Our results suggest that country and language cues should be evaluated as distinct signals rather than interchangeable proxies for culture.