Language aware model reduces errors in code switching speech recognition
LLM-HB: Language-Aware LLM-Guided Hotword Biasing for Code-Switching ASR
SoundMachine Learning
Summary
Speech recognition systems struggle when people switch languages in the middle of talking, which happens a lot in multilingual communities. The authors created a new method called LLM-HB that uses a large language model to focus on key words and predict which language is being spoken. This approach helps the system understand the mixed language better and reduces errors. Tests showed it cuts recognition mistakes by about 20% on a challenging dataset.
What this means in practice
- •For multilingual speech app developers: Improve speech recognition accuracy in apps that need to handle users switching languages mid-utterance by using LLM-guided keyword biasing techniques.
- •For voice assistant engineers: Enhance voice assistants in multilingual markets to better understand mixed-language commands by integrating language-aware hotword biasing.
Tested on one dataset.
Authors
Yuxuan He, Genshun Wan, Pengcheng Li, Gongping Huang, Jianqing Gao, Yanmin Qian
Abstract
Code-switching (CS) speech, characterized by language alternation within a single utterance, is prevalent in multilingual communities and poses challenges to automatic speech recognition (ASR) due to rapid language transitions. We propose LLM-HB, a language-aware large language model-guided hotword biasing framework for code-switching ASR (CS-ASR). The framework incorporates a mixture-of-experts (MoE) adaptor and an auxiliary language prediction head to better capture and differentiate multilingual representations, while hotwords are injected through LLM-based prompting. Experiments on the ASRU2019 code-switching challenge test set demonstrate that LLM-HB achieves a 20.30% relative reduction in mixed error rate (MER) over the baseline, reaching 5.85% MER under a 15-distractor hotword setting. To the best of our knowledge, this is the first work to integrate LLM-guided biasing into code-switching ASR. We further release the extracted ASRU2019 hotword list to support future research.