Progressive Refinement: An Iterative Pseudo-Labeling Approach for Mandarin-English Code-Switching ASR
2026-07-06 • Computation and Language
Computation and Language
AI summaryⓘ
The authors studied how to improve speech recognition when people switch languages mid-sentence, which is hard because not much training data exists for this. They used a method called iterative pseudo-labeling, where a model guesses labels for unlabeled speech data to create more training examples. Then, they trained the model in two steps: first with this larger data, then fine-tuning it with actual labeled data. By repeating this process, their system better understands mixed-language speech and reduces errors significantly on test sets.
code-switchingautomatic speech recognitionpseudo-labelingsemi-supervised learningbilingual modelmixed error rateiterative trainingSEAME dataset
Authors
Qu Yang, Cakra Wardhana, Tim Ng
Abstract
Code-switching (CS), alternating languages within the same utterance, poses significant challenges for automatic speech recognition (ASR) due to limited CS training data. This paper applies an iterative pseudo-labeling training approach to CS-ASR for the first time, demonstrating its effectiveness in leveraging unlabeled data to improve CS-ASR performance. The approach comprises three phases: pseudo-label generation, two-stage bilingual model training, and iterative improvements. It begins by generating pseudo-labels from a large unlabeled corpus, creating a semi-supervised dataset. This dataset supports a two-stage training framework where the model is pre-trained and then fine-tuned on supervised CS data. Iterative refinements further enhance the model's accuracy in handling complex CS scenarios. Our approach significantly advances CS-ASR systems, achieving notable Mix Error Rate (MER) reductions on SEAME's devman (6.35%) and devsge (8.29%) subsets.