Speechllm improves domain term recognition by selective dictionary retrieval
Training-Free Contextual ASR via SpeechLLM-Based Error-Aware Selective Retrieval
Sound
Summary
Recognizing special or uncommon words in speech, like medical or financial terms, is tough for speech recognition systems. The authors designed a method that first guesses the transcription and spots where errors might happen using a big speech language model. Then, only those parts are used to look up similar-sounding words from a special dictionary. Finally, the system re-listens to the audio with help from these suggestions to get a better transcription. This approach works well across different fields without needing extra training.
What this means in practice
- •For medical transcription teams: Improve recognition of complex medical terms by targeting dictionary lookups to likely error spots for better transcription accuracy.
- •For air traffic control operators: Enhance transcription of low-frequency air traffic terms by using error-aware retrieval to reduce irrelevant dictionary queries.
Authors
Natsuo Yamashita, Ai Nemoto, Ryosuke Koichi, Masaaki Yamamoto
Abstract
Recognition of domain-specific and low-frequency terms remains challenging for automatic speech recognition (ASR). Although contextual biasing can improve their recognition, directly providing a large terminology dictionary introduces many irrelevant biasing terms. Retrieval-based contextual biasing addresses this issue by selecting candidate terms from an external dictionary, but querying many recognized words requires numerous dictionary lookups and may yield poorly targeted candidates. We propose a training-free contextual ASR framework in which a pretrained speech large language model (SpeechLLM) jointly generates an ASR hypothesis and localizes error spans likely to involve domain-specific terms. Only the localized spans are used to retrieve phonologically similar terms from an external terminology dictionary. The same SpeechLLM then re-recognizes the audio conditioned on the first-pass hypothesis and the retrieved terms, without task-specific model training. To assess applicability across domains, we evaluate the framework on medical, air traffic control, and financial speech. The proposed method substantially reduces dictionary queries while improving the recall and ranking of relevant terminology candidates and second-pass ASR performance across all three domains.