AI summaryⓘ
The authors address the challenge of building speech recognition models that can understand hundreds of languages without losing accuracy. They create a system called MoLGE, which groups similar languages together and assigns special expert modules to each group, making the model more efficient. They also add a method called Low-Rank Adaptation to better capture language-specific sounds and words without adding too many extra parameters. Their experiments showed that MoLGE performs better than traditional models on 495 languages, especially by improving both how the model hears sounds and recognizes written language. The authors conclude that grouping languages thoughtfully helps improve large multilingual speech systems.
automatic speech recognition (ASR)multilingual modelsMixture of Experts (MoE)self-supervised models (S3Ms)Low-Rank Adaptation (LoRA)language groupingphoneticsorthographyparameter efficiencylanguage scalability
Authors
Sangmin Lee, Woojin Chung, Woongjib Choi, Hong-Goo Kang
Abstract
Massively multilingual automatic speech recognition (ASR) models covering hundreds of languages must maintain robust performance across diverse linguistic and acoustic conditions. However, these models often encounter the curse of multilinguality, where model capacity is diluted across languages. To address this challenge, we propose Mixture of Language Group Experts (MoLGE), built upon speech self-supervised models (S3Ms). MoLGE assigns dedicated expert modules to clusters of similar languages, reducing the number of required submodules compared to conventional language-specific Mixture-of-Experts (MoE) schemes. It further integrates a hierarchical Low-Rank Adaptation (LoRA) strategy into the disentangled acoustic and linguistic components of the S3M architecture, enabling efficient modeling of language-specific characteristics while maintaining parameter efficiency. Further, we investigate the impact of language grouping strategies based on both linguistic and data-driven criteria on overall performance, providing an interpretable perspective on how language structure influences scalability in multilingual speech systems. In experiments, we evaluate MoLGE on a multilingual benchmark encompassing 495 languages. Results demonstrate that MoLGE consistently outperforms dense multilingual baselines with a minimal increase in trainable parameters. Notably, these language grouping strategies yield substantial improvements for both phonetic and orthographic aspects of ASR modeling. Our findings suggest that structured language specialization provides an effective pathway for massively scaling language coverage of multilingual ASR.