Group relative policy optimisation improves speech recognition in low-resource languages

Group Relative Policy Optimisation Improves Multilingual Speech Recognition in Low-Resource Languages

Artificial IntelligenceComputation and Language

Summary

Speech recognition systems often struggle to correct their own mistakes when converting spoken words to text, especially for languages with limited training data or noisy audio. The authors showed that a method called Group Relative Policy Optimisation (GRPO), which uses reinforcement learning, can help smaller speech models perform better on many African languages. Their improved model, SunflowerASR, achieves the best reported accuracy on two benchmarks for African languages, particularly when data is scarce or audio quality is poor. This approach could help make speech recognition more reliable across diverse languages with fewer resources.

What this means in practice

  • For speech recognition developers: Build better speech-to-text models for underrepresented languages using reinforcement learning to reduce errors from scarce data or noisy inputs.
  • For language technology product teams: Improve voice interfaces and transcription services for African languages by deploying a more accurate speech recognition backbone.$Commercial implications: Enables new or enhanced commercial speech-enabled products targeting African language speakers with fewer data resources.

Authors

Tim Wenjie Hu, Evelyn Nafula Ouma, Benjamin Akera, John A. Quinn

Abstract

Autoregressive encoder-decoder speech recognition models trained with teacher forcing can suffer from exposure bias, limiting their ability to recover from their own prediction errors at inference time. Recent work has shown that reinforcement learning with Group Relative Policy Optimisation (GRPO) improves speech recognition in major languages with LLM-based models. We extend this line of work to smaller, non-LLM encoder-decoder ASR models, using Whisper-large-v3 as a case study across 51 African languages. The benefits of this approach are most pronounced when training data is scarce or when input audio is noisy or degraded. Our resulting model, SunflowerASR, achieves the best published macro-averaged WER on the AfriVox-v2 (0.227) and SimbaBench (0.372) benchmarks, and is publicly available.
View PDFOpen arXiv