Self supervised models reduce accent confusion in spoken language id
Mitigating Accent-Language Confusion in Self-Supervised Speech Representations for Language Identification
Computation and LanguageSound
Summary
Spoken language identification systems sometimes mistake a person's accent for a different language, especially when the speaker is not using their native accent. The authors found that the speech features from non-native speakers fall between their first language and the target language, causing errors. They created a way to adjust these features by removing bias linked to the speaker's first language without needing examples of non-native speech. This approach improved the system's ability to correctly identify languages spoken with different accents while keeping accuracy high for native speakers.
What this means in practice
- •For speech recognition developers: Reduce misclassification of languages caused by accented speech in voice assistants and transcription services.
- •For telecommunication service teams: Improve language identification accuracy for global users speaking with diverse accents to better route calls or customize services.
Authors
Minu Kim, Jihwan Lee, David R. Mortensen, Shrikanth Narayanan
Abstract
Spoken language identification (LID) aims to recognize the target language regardless of accent. In practice, however, LID models fine-tuned from self-supervised speech representations frequently confuse accents with languages, misclassifying non-native (L2) speech as the speaker's first language (L1). We show that non-native speech representations lie between native target-language and native L1 poles, causing systematic misclassification. To address this, we introduce a geometric projection that estimates an L1-bias direction solely from native speech and removes it before the frozen LID head. Across five MMS-LID models and non-native corpora, this projection substantially improves target language identification for L2-accented speech while preserving predictions for native speech. These results show that accent-induced L1 bias can be corrected directly within the representation space without L2 training data or model adaptation.