Large language models struggle with diverse narratives across languages

The Narrative Gap: Can LLMs Help Us Navigate Diverse Narratives Across Languages?

Computation and LanguageArtificial IntelligenceHuman-Computer InteractionInformation Retrieval

Summary

People get information from many sources, but stories can differ a lot depending on where and in what language they are told. The authors studied if large language models (LLMs) can help people find and understand these different stories on the same topics across languages. They created a new test called NarrativeBench with news and questions in many languages to check how well LLMs do. They found that current models often miss or hide some viewpoints, making it harder for users to see the full picture and risking more misunderstandings. This work helps build better tools to explore and compare different stories worldwide.

What this means in practice

  • For multilingual search engineers: Improve search engines to offer users balanced viewpoints by detecting and surfacing diverse narratives in different languages on controversial topics.
  • For social media platform teams: Design tools that help users explore competing narratives and reduce linguistic echo chambers by highlighting missing or excluded viewpoints across languages.

Authors

Nikhil Sharma, Kelly Marchisio, Kenton Murray, Ziang Xiao

Abstract

Large language models (LLMs) increasingly mediate how people seek and consume information, including accounts that differ across languages and regions, especially on contested topics. Supporting diverse information seeking requires models to help users navigate competing narratives across languages and information streams. We define six capabilities necessary for diverse multilingual information seeking: three foundational capabilities for narrative inclusion, narrative exclusion, and missing narrative identification; and three higher-level capabilities for presenting diverse narratives, preserving narrative coverage under confirmatory queries, and maintaining information parity across languages. To guide the development of those information seeking capabilities, we introduce \textsc{NarrativeBench}, a benchmark comprising 1,333 news articles in 23 publication languages across 23 contested topics and 460 Reddit-grounded queries. Evaluating 14 LLMs across 24 query languages, we find that current models lack the capabilities needed to reliably navigate diverse narratives, limiting users' control over the perspectives they receive and leaving substantial gaps in exposure to competing accounts even when diversity is explicitly requested. Information access also differs across query framings and languages, risking echo chambers, linguistic filter bubbles, and information disparities that weaken the common ground needed for democratic discourse. \textsc{NarrativeBench} provides a foundation for developing information systems that give users across languages and regions agency to navigate diverse narratives.