BuzzASR improves speech recognition for 102 individual languages
BuzzASR: A Swarm of 100+ Monolingual Speech Recognition Models
Computation and Language
Summary
Speech recognition systems that handle many languages at once often don’t work well for less common languages. The authors created BuzzASR, a group of speech recognition models each fine-tuned specifically for one language. This approach greatly improves accuracy in recognizing speech for 102 languages by adjusting the model and its text processing part for each language. BuzzASR outperforms the original multilingual model on most tested languages and is publicly available for use.
What this means in practice
- •For speech technology developers: Build more accurate language-specific speech recognition systems by fine-tuning large multilingual models with adapted tokenizers and monolingual data for many new languages.
- •For accessible technology providers: Provide improved voice interfaces and transcription services for speakers of less-supported languages using BuzzASR’s specialized models.
Authors
Shivam Singh, Aditya Yadavalli, Catherine Arnett, Alex Warstadt
Abstract
We introduce BuzzASR, a collection of language-specialized fine-tuned Whisper models adapted for automatic speech recognition (ASR) in 102 languages. Large end-to-end Transformer-based ASR models such as Whisper have revolutionized ASR, but most prominent models are highly multilingual. As a result, these models often perform poorly on languages less well-represented in their training set. While it has long been known that effective language adaptation can be achieved through simple fine-tuning on monolingual data, this strategy has only been applied to a small number of languages. We massively scale up this simple approach to 102 languages covered in the FLEURS dataset, while also implementing a more complex language adaptation strategy that integrates monolingual tokenizer replacement and data augmentation using text-only fine-tuning. BuzzASR models outperform Whisper-large-v3 on 77 out of 102 languages, reducing character error rates (CER) by a factor of over 2.8 on average. Our models achieve state-of-the-art CER among open-source systems on 27 of 102 languages on the combined FLEURS and Common Voice test set. Our tokenizer replacement strategy yields an average 3.3x improvement in compression rate (characters per token) over Whisper's multilingual BPE, with gains of up to 21.7x. We release all models, code, and detailed results: https://lemn-lab.github.io/buzz-asr