Phoneme level method improves rare word recognition in speech models
PTC-Bias: Phoneme-Level Temporal Competition for Bias Retrieval and Post-Decoding Correction in Speech LLMs
Computation and LanguageSound
Summary
Speech recognition systems often struggle to correctly hear rare or unusual words. The authors developed a two-step method called PTC-Bias that listens to small sounds (phonemes) over time to guess likely rare words before and after the main speech-to-text step. This approach narrows down possible confusing words and fixes mistakes without slowing down the main model. Tests show it reduces errors on commonly used speech datasets while keeping other words just as accurate.
What this means in practice
- •For speech recognition developers: Integrate PTC-Bias to improve accuracy for rare or unusual words in speech-to-text services without extra computational cost.
- •For voice assistant engineers: Enhance voice-controlled products’ understanding of uncommon names or terms by using phoneme-level biasing for real-time correction.$Commercial implications: Enables more reliable recognition features in commercial voice assistants that handle personalized and extensive vocabularies.
Authors
Zhiqi Ai, Han Cheng, Shiyi Mu, Yongjin Zhou, Shugong Xu
Abstract
Contextual biasing improves rare-word recognition in speech large language models (SpeechLLMs), but efficiently exploiting large bias lists remains challenging. We propose PTC-Bias, a two-stage framework based on phoneme-level temporal competition. At the prefill stage, PTC Retrieval performs frame-synchronous phoneme decoding and temporal competition among candidate pronunciations, producing a compact bias-word shortlist and corresponding speech intervals. After SpeechLLM decoding, PTC Correction conducts a second local competition between the retrieved candidates and mismatched transcript spans within these intervals. Selective correction reduces near-homophone and word-segmentation errors while preserving correct transcriptions. Both stages share the same phoneme posteriors and require no additional SpeechLLM forward pass. Experiments on LibriSpeech show consistent gains across two SpeechLLMs and bias lists of up to 2000 words. With Prompt-SLAM-ASR-7B and 2000 bias words, PTC-Bias reduces B-WER by 23.4%/23.9% relative to CTC-Filter on test-clean/test-other, while keeping U-WER nearly unchanged.