Large language models struggle to detect culture specific stereotypes accurately
Cultural (In)Competence: Investigating Culture-Specific Stereotype Detection Capabilities of LLMs
Computation and Language
Summary
Stereotypes depend a lot on culture, so understanding them needs knowledge about cultural differences. The authors tested how well large language models (LLMs) can spot stereotypes specific to US, Dutch, and Chinese cultures. They found these models often say a stereotype is present when it is not, especially if the prompt language is English. Using prompts in other languages and adding extra training helped the models be less wrong about stereotypes. Bigger training datasets helped too, but not always in a simple way.
What this means in practice
- •For multilingual chatbot developers: Improve chatbot fairness by adjusting language and instruction tuning to better handle culture-specific stereotype detection.
- •For content moderation teams: Design moderation systems that reduce false positives by considering prompt language and fine-tuning data sizes when detecting stereotypes.
Authors
Maximilian Spliethöver, Jannes Lötfering, Dominik Bachmann, Alexander Espig, Willem Zuidema, Katrin Schulz, Henning Wachsmuth
Abstract
Stereotypes are subjective constructs that are mediated by the cultural context in which they are viewed. Successfully modeling them thus also requires cultural competence. When using large language models (LLMs) for stereotype-related tasks, such as their detection, it is therefore crucial that models can access relevant cultural knowledge and adjust responses accordingly. Since LLMs have shown disparities in what cultural knowledge is available to them, their capacity regarding stereotypes may also be disparate across languages and cultural contexts (e.g., for non-English languages or for non-US contexts). In this paper, we systematically investigate the effectiveness of LLMs for culture-specific stereotype detection across three factors: (1) the prompting language, (2) the instruction tuning language, and (3) the fine-tuning dataset size. We conduct experiments for US, Dutch, and Chinese cultural contexts. Our results suggest that LLMs do not reliably account for cultural contexts and overpredict the presence of stereotypes. Non-English prompts seem to improve consideration of cultural context and thus lead to more effective predictions. Moreover, we find that additional instruction tuning reduces false-positive rates and that training data size moderates detection effectiveness non-monotonically.