Papers for

multilingual app developers

Papers whose findings have a practical use for this group, as judged from the abstract. Open a paper to read what it means in practice.

Language and country cues affect large language model cultural views differently

Investigating Multi-Cultural Value Selection Biases in LLMs

Abstract: Large language models (LLMs) serve users whose cultural identities are multi-layered, yet cultural evaluations commonly reduce identity to one label. In this work, we study LLMs under simultaneous identity cues, with a country persona in the system prompt and a second identity signaled in the user turn, either by naming a country (explicit cue) or through the query language (implicit cue). We test World Values Survey (WVS) multiple-choice probing and open-ended sociopolitical generation. In multiple-choice probing, language cues shift answers about three times more than explicit cues but away from the associated country's distribution, and explicit cues do not consistently move answers toward the named country. Across 1.6 million open-ended responses, naming a country closes 30-44% of the gap to the model's own responses for that country, and explicit cues produce more differentiated value changes than language cues. Our results suggest that country and language cues should be evaluated as distinct signals rather than interchangeable proxies for culture.

Thu 22 OctComputation and Language
The gist
Large language models are asked to reflect cultural values, but they often rely on just one cultural label when people have multiple identities. The authors studied how these models respond when given two cultural clues: a country and a language. They found that language cues change the model’s answers more than just naming a country, but not always in a way that matches that country’s typical views. This means that language and country labels should be treated as separate signals in how these models understand culture.
Open → 2610.31199v1

Training-free steering improves speech model task control with vectors

Steering Speech-Language Models: Training-Free Task Specialization via Contrastive Activation Addition

Abstract: Activation steering has proven effective for controlling the behavior of Large Language Models (LLMs) at inference time, but its application to SpeechLLMs remains new, and training-free steering approaches for such models are still largely unexplored. We propose a training-free Contrastive Activation Addition (CAA) protocol that derives steering vectors for common speech tasks (e.g. transcription) in SpeechLLMs from a small number of labeled utterances. We showcase that adding these vectors in the representation space, at inference time, enforces better the targeted speech task. We further show that, when combined with prompting, these vectors yield to consistent improvement over prompting alone on most evaluated tasks such as Automatic Speech Recognition (ASR) or Emotion Recognition (ER), and transfer to out-of-domain data. We additionally demonstrate the usefulness of script-normalization directions to enforce the target script of a specific language.

Sat 3 OctComputation and Language
The gist
Making speech recognition and understanding tools better usually requires retraining big models, which takes time and resources. This paper shows a way to guide these speech models to focus on specific tasks without retraining by adding special adjustment signals called steering vectors. These vectors are built from just a few examples and help the model perform tasks like transcribing speech or recognizing emotions more accurately. The approach works well even on new data types and can control the script used in different languages.
Open → 2610.04683v1

Semantic scaffold method improves summary evaluation beyond current metrics

Preserving What Matters: Semantic Scaffolds Beyond Saturation in Summarization Evaluation

Abstract: Summarization ships in countless production systems, making model selection a routine decision that depends on measuring summary quality. Existing metrics struggle to support this: ROUGE captures only surface overlap, while LLM-as-judge scores saturate to near-identical values that fail to rank models effectively. We observe this saturation across three public datasets, two proprietary datasets, and multilingual settings. Motivated by this, we introduce Semantic Scaffold, an evaluation framework that extracts a hierarchical representation of facts, questions, and entity attributes from a source text, labeling each as a main point or supporting detail, and reusing this structure as a fixed reference for scoring summaries. From this representation, we derive three diagnostic metrics: Fact Preservation Score (FPS), Question Preservation Score (QPS), and Entity Preservation Score (EPS), designed to reward the preservation of essential information while penalizing detail overload, and position them as interpretable diagnostics that remain informative where holistic axes collapse. Finally, we analyze four recurring failure modes of ROUGE and LLM-as-judge scores, demonstrating that scaffold-based evaluation remains informative where conventional metrics collapse.

Fri 18 SeptComputation and LanguageArtificial Intelligence
The gist
Measuring how good a summary is can be tricky because common tools either only check for word overlap or give nearly identical scores for very different summaries. The authors noticed this problem happens in many different data sets and languages. They created a new way called Semantic Scaffold that breaks down the important facts, questions, and details in the original text and checks if these key parts are kept in the summary. Their new scores can better tell which summaries keep the important information without overwhelming detail, even when old methods fail.
Open → 2609.22603v1