Frozen gabor kernels improve speech recognition for many languages
Orukeet: Multilingual ASR with Frozen Gabor Kernels
SoundMachine Learning
Summary
Speech recognition systems try to understand spoken words from audio recordings, but doing this well across many languages and accents is challenging. The authors replaced part of an existing speech recognition model with special filters called Gabor kernels, which they fixed so they wouldn’t change during training. This approach helped the system understand speech better in 25 languages and multiple accents, reducing error rates consistently. Their improved model works within the same overall system, so it can be used just like before but with better accuracy.
What this means in practice
- •For speech technology developers: Enhance multilingual speech recognition software by integrating frozen Gabor kernels for better accuracy across many accents and languages.
- •For call center automation teams: Improve voice recognition systems for multi-accent customer interactions by using the proposed model adjustments.
Authors
Nathan Roll, Irene Yi, Büşra Marşan, Vianney Grenez, Gabriel Stein, Momcilo Mrkaic, Pavle Padjin, Vladimir Zeljkovic, Calbert Graham
Abstract
Orukeet replaces half of an adapted Parakeet encoder's temporal filters with 12,288 fitted Gabor kernels, freezes these replacements, and trains the remaining parameters on multilingual and multi-accent data. Final adaptation and checkpoint selection use LibriSpeech test-other. Across 20,146 FLEURS recordings in 25 languages, pooled word error rate (WER) falls from Parakeet's 11.01% to Orukeet's 9.85%, a 10.6% relative reduction. Orukeet has lower WER on 23 of the 25 languages. Orukeet outperforms Parakeet on 61 out of 74 tested splits, including LibriSpeech test-clean (1.46% vs. 1.53% WER), test-other (2.86% vs. 3.14%), and FLEURS English (3.82% vs. 4.28%). All comparisons decode the same audio with matched NeMo settings. The fitted kernels are stored as ordinary convolution weights, retaining Parakeet's architecture and inference operators.