Papers for
multilingual ai product teams
Papers whose findings have a practical use for this group, as judged from the abstract. Open a paper to read what it means in practice.
Chinese version of empathy scale for AI systems shows cultural differences
A Systematic and Validated Translation of the Perceived Empathy of Technology Scale to Chinese
Abstract: The Perceived Empathy of Technology Scale (PETS) allows evaluation of empathic behavior in intelligent systems, a concept increasingly applied in human-AI interaction, particularly in emotional and mental health support applications. To support intercultural research and development, PETS is already available in multiple languages, as research suggests cultural and language-based differences in the expression and perception of empathy. Given the substantial Chinese-speaking population and the growing focus on cross-cultural HCI research, we present a validated translation of PETS into Putonghua (Mandarin) with Simplified Chinese. Based on established back-translation methodologies as used in previous PETS translations, our process included multiple independent translations, expert panel reviews, and empirical testing with a total of N=400 participants from an English and a Chinese sample. Through confirmatory factor analysis and invariance testing we show that the Chinese PETS retains the original two-factor structure with high reliability, with support for configural, metric, and scalar invariance. Beyond validation, empathic system behavior was rated equivalently across languages, while non-empathic behavior drew substantially higher ratings from Chinese participants, suggesting its absence is read less consistently across cultures than its presence. This validated Chinese version enables researchers and practitioners to assess empathy perceptions among Chinese-speaking users, supporting culturally appropriate empathic technology development.
Speech benchmark exposes gaps in Southeast Asian language AI
SEA-SpeechBench: A Large-Scale Multitask Benchmark for Speech Understanding Across Southeast Asia
Abstract: The rapid advancement of audio and multimodal large language models has unlocked transformative speech understanding capabilities, yet evaluation frameworks remain predominantly English-centric, leaving Southeast Asian (SEA) languages critically underrepresented. We introduce SEA-SpeechBench, to the best of our knowledge, the first large-scale multitask benchmark that evaluates speech understanding in 11 SEA languages through 97,194 samples across 99 evaluation sets and 597 hours of curated audio data. Our benchmark comprises 9 diverse tasks across 3 categories: speech processing (automatic speech recognition, speech translation, spoken question answering), paralinguistic analysis (emotion, gender, age, speaker recognition), and temporal understanding, a novel dimension featuring timestamped content queries and temporal localization within extended audio sequences up to 3 minutes. We implement multilingual prompting in both native SEA languages and English to reflect user interactions with audio-language models. Evaluation of leading open-source and proprietary systems reveals marked performance gaps. Across all models, performance remains underwhelming on temporal understanding, emotion recognition, and speech translation. Prompting in low-resource languages such as Burmese and Tamil lags behind English by up to 41 percentage points. Our findings expose critical model limitations and underscore the need for inclusive model development. The SEA-SpeechBench benchmark is available at https://zwenyu.github.io/SEA-SpeechBench/.