Speech recognition boosted by context using homophone clues
HearInContext: A Benchmark for Implicit Context in Speech Recognition
Computation and LanguageSound
Summary
Speech recognition systems sometimes get confused by words that sound the same but have different meanings. The authors created a test set using Mandarin and English homophones to see how well models understand hidden context in conversations. They found that systems get better at recognizing the right word when given subtle clues, and do even better when the correct word is explicitly mentioned. By training a large speech model, they improved its ability to pick the correct word without hurting its general speech recognition accuracy.
What this means in practice
- •For voice assistant developers: Improve voice assistants to better understand spoken commands involving homophones or ambiguous words using subtle conversational context.$Commercial implications: Enables more accurate assistant responses by improving speech recognition on contextual word ambiguities, enhancing user experience.
- •For speech recognition engineers: Use the HearInContext benchmark to evaluate and enhance speech models’ ability to use implicit context for better word recognition accuracy.
Authors
Yifan Gao, Yao Tian, Hongbin Suo
Abstract
Contextual ASR can benefit from semantic cues or from target words explicitly provided in the context. We introduce HearInContext, a Mandarin--English benchmark that pairs shared synthetic speech with assistant replies supporting different interpretations. The benchmark comprises 3,764 semantic test cases built around homophones. Implicit contexts exclude candidate words; explicit contexts name the target. No-context and unrelated-context controls measure the benefit of relevant history and sensitivity to irrelevant history. Context-capable models benefit from implicit cues but achieve higher target recall with explicit hints. Fine-tuning Qwen3-ASR-1.7B improves implicit-context target recall by 11.0 and 11.5 percentage points in Mandarin and English, respectively, while absolute CER/WER changes on AISHELL-1 and LibriSpeech remain below 0.1 percentage points. Gains extend to explicit conditions excluded from fine-tuning and to Mandarin hotword recognition on real recordings.