Evaluating and Mitigating Anti-LGBTQ Biases in German and Multilingual Language Models
2026-08-31 • Computation and Language
Computation and LanguageArtificial Intelligence
AI summaryⓘ
The authors created a new test in both German and English to check if language models show bias against LGBTQ people, especially focusing on cultural and language differences. They used stereotypes shared by queer individuals in German-speaking communities and a translated existing dataset. After testing eight different language models, they found these models often repeat anti-LGBTQ stereotypes, though results vary by model and specific identities. They also tried improving the models by training them on positive community content, which helped sometimes but not always. Their work shows the need to consider culture and language when measuring biases in language technology.
language modelsanti-LGBTQ biasmultilingual benchmarksstereotypesfine-tuningWinoQueercultural adaptationqueer communitybias evaluationGerman-English datasets
Authors
Melina Morch, Daniel Braun
Abstract
While gender and racial biases in language models have been widely studied, anti-LGBTQ biases remain underexplored, particularly beyond English. Existing benchmarks often do not capture cultural and linguistic variation and rely on gender representations. This paper introduces a multilingual German-English benchmark dataset for the evaluation of anti-LGBTQ biases in language models. It combines community-sourced stereotypes from German-speaking queer individuals with a German translation of WinoQueer. The data is used to evaluate eight language models across sizes and architectures and explore mitigation through fine-tuning on community and progressive media content. Results show that language models reproduce anti-queer stereotypes, with variation across identities and models. Differences between the translated and community-based data highlight the importance of cultural adaptation for multilingual bias evaluation. Fine-tuning reduces bias on average, but not consistently across models and identities. Warning: This text contains examples of anti-queer hateful language and stereotypes.