Using pseudo-labels as rewards improves speech recognition accuracy
Pseudo-Labels as Rewards, Not Targets: Group-Relative Policy Optimization for Semi-Supervised ASR
Computation and Language
Summary
Training speech recognition systems usually relies on labels telling the model what was said, but getting these is expensive. People create 'pseudo-labels' by using an existing system to guess the words, then teach a new system to copy those guesses. The problem is if the guesses contain errors, the new system simply learns those mistakes. The authors show a new way to use these guesses not as exact answers but as rewards that guide learning, which helps the new system avoid copying errors and improves accuracy.
What this means in practice
- •For speech technology engineers: Improve domain adaptation of speech recognition systems using noisy unlabeled audio without increasing label errors.
- •For multilingual voice assistant teams: Enhance transcription accuracy across multiple languages and domains by leveraging reward-based learning from existing model outputs.
Authors
Shashi Kumar, Yanis Labrak, Sergio Burdisso, Esaú Villatoro-Tello, Petr Motlicek, Kadri Hacioglu, Andreas Stolcke
Abstract
Adapting ASR to a domain with only untranscribed audio typically relies on pseudo-labeling, a strong teacher transcribes the audio and a student is fine-tuned on it. Because supervised fine-tuning (SFT) treats every pseudo-label token as a target, the student inherits the teacher's errors. We show that using the same pseudo-labels as a reward reference for Group Relative Policy Optimization (GRPO) is more robust to label noise. In the banking domain, GRPO yields 29.3% to 56.2% relative WER improvements over SFT on Whisper-generated labels, and the advantage holds across four Whisper sizes, four teacher families, three domains and four languages. While SFT reproduces almost 100% of teacher errors verbatim, GRPO passes through only 27%. Rollout statistics explain why, erroneous pseudo-label words rarely appear in on-policy rollouts, so group-relative advantages stay aligned with the clean transcript, whereas SFT forces them into the policy's support and collapses the reward variance GRPO relies on.