BranchShine: Compact Raw-Audio-to-IPA Transcription with a RoPE E-Branchformer Encoder
2026-06-22 • Machine Learning
Machine Learning
AI summaryⓘ
The authors created BranchShine, a small speech-to-IPA transcription model that converts spoken audio directly into pronunciation symbols. Despite having far fewer parameters than a much larger baseline model, BranchShine achieves nearly the same accuracy on a test set involving 41 languages. They also found that BranchShine is better at avoiding mistakes with children's speech, while another model, Whisper-Medium, is better at exact correct matches. This shows that smaller models can still do well on detailed speech transcription tasks.
Speech-to-IPA transcriptionCTC recognizerRoPE E-BranchformerCharacter error rateMultilingual speech recognitionRaw audio modelPhonetic transcriptionWhitespace-insensitive scoringConvolutional neural networksChild speech recognition
Authors
Nikhil Navas, Sergio Chevtchenko, Talisson Damiao, Saeed Afshar
Abstract
Speech-to-IPA transcription is useful when the desired output is pronunciation rather than orthographic text, but competitive multilingual systems are often large and evaluation is sensitive to normalization choices. This paper presents BranchShine, a 33M-parameter raw-audio CTC recognizer with a lightweight convolutional front end and a 19-block RoPE E-Branchformer encoder. We find that BranchShine provides a compact and competitive operating point for IPA transcription under matched normalization and scoring. On a 16,660-utterance multilingual test set covering 41 language labels, BranchShine obtains 9.19% whitespace-insensitive IPA character error rate, compared with 9.78% for the 575.00M-parameter PhoneticXEUS baseline. A secondary child speech reading analysis shows a complementary operating profile: BranchShine is more conservative on incorrect readings, while Whisper-Medium is stronger on exact acceptance of correct readings. Overall, the results indicate that a compact raw-audio-to-IPA model can approach much larger baselines on character-level IPA transcription.