Far-field speech recognition benchmark uses high-quality simulated room effects
FFASR: Benchmarking Far-Field Automatic Speech Recognition using High-Fidelity Simulated RIRs
SoundArtificial IntelligencePerformance
Summary
Speech recognition systems work well when microphones are close but struggle with far-away speakers, especially in noisy, echo-filled rooms. The authors created a new test collection called FFASR that uses simulated room echoes and noise to evaluate recognition accuracy in realistic far-field conditions. They found that simulated room effects closely match real recordings and significantly impact recognition errors. This benchmark helps guide development of speech recognition systems for far-field and moving speakers.
What this means in practice
- •For voice assistant developers: Test and improve far-field speech recognition models by using a reliable simulated dataset that mimics real room reverberations and noise conditions.
- •For smart speaker manufacturers: Validate smart speaker performance in various noisy and reverberant room scenarios without needing extensive real-world recordings.$Commercial implications: Enables product testing for diverse acoustic conditions improving smart speaker market readiness.
Authors
Shivam Saini, Eric Bezzam, Georg Götz, Alessia Milo, Steinar Guðjónsson, Konstantinos Gkanos, Finnur Pind, Daniel Gert Nielsen
Abstract
Far-field automatic speech recognition(ASR) degrades under reverberation, noise, and talker motion, yet the benchmarks that drive model selection emphasize close-microphone speech. We present FFASR, a held-out corpus of 15,637 utterances and an open leaderboard spanning nine conditions, each varying a single acoustic factor: anechoic near-field speech, a measured-versus-simulated office-lab pair, static far-field mixtures at high/mid/low signal-to-noise ratio(SNR), and moving-talker variants at matched SNR. Dry speech from 15 talkers is convolved with hybrid wave/geometrical-acoustics room impulse responses from 14 furnished rooms; because the speech is newly recorded and the test waveforms are never released, the corpus resists training-data contamination. Across contemporary systems, mean word error rate (WER) rises from 4.4% near-field to 41.3% in the static low-SNR condition; a moving talker adds a small but consistent penalty at matched SNR; and on the office-lab pair, measured and simulated WER agree to within about 1.7 pp on average. These results support high-fidelity simulation as a scalable proxy for measured far-field evaluation under the conditions we test.