Low-resource multilingual text-to-speech systems struggle with complex texts
Complex-Text Robustness Evaluation and Failure Diagnosis for Low-Resource Multilingual Text-to-Speech
Computation and LanguageSound
Summary
Text-to-speech systems that work in many languages but with limited data often have trouble reading complicated texts correctly. The authors created a way to test how well these systems handle tricky inputs like numbers, dates, and mixed languages. They introduced a simple score to predict when the system might make mistakes before actually generating speech. Testing on several popular systems showed specific weaknesses that normal tests don’t catch. This new evaluation method helps find and understand failures in multilingual speech synthesis.
What this means in practice
- •For speech technology developers: Diagnose weaknesses in multilingual TTS systems against complex inputs like code-switching and numbers for improved system robustness.
- •For localization engineers: Identify risky text inputs automatically before synthesis to reduce errors when generating speech in multiple low-resource languages.
Authors
Tianlun Zuo, Ziyu Zhang, Tingzhi Mao, Zhonghua Fu, Lei Xie
Abstract
Low-resource multilingual text-to-speech (TTS) systems have expanded language coverage, but their robustness under complex text inputs remains insufficiently diagnosed. Existing evaluations mainly focus on naturalness, speaker similarity, and content consistency using regular test sentences, while providing limited insight into how multilingual TTS systems fail when handling challenging inputs such as numbers, dates, named entities, long sentences, code-switched expressions, and punctuation-related structures. This paper proposes a complex-text robustness diagnosis framework for low-resource multilingual TTS. We evaluate robustness from three dimensions: content consistency, language consistency, and generation stability. A multilingual robustness testing scheme is designed for Thai, Vietnamese, Swahili, and Indonesian, covering ordinary sentences and multiple types of complex text inputs. We further introduce automatic diagnostic metrics, including character error rate, language identification accuracy, and duration abnormal rate. To support input-level risk analysis before speech generation, we propose a lightweight Text Risk Score (TRS), which estimates synthesis risk from interpretable text features without manual annotation or model training. Experiments on three representative multilingual TTS systems, including OmniVoice, VoxCPM2, and MMS-TTS, show that complex text inputs expose systematic failure patterns that are not fully reflected by ordinary short-sentence evaluation. Different systems exhibit distinct vulnerabilities in number normalization, named entity handling, long-text generation, and code-switched input processing. Furthermore, TRS shows a positive correlation with content errors and duration abnormalities, demonstrating its usefulness as a low-cost pre-synthesis indicator for complex-text risk diagnosis in low-resource multilingual TTS.