Low resource speech recognition improves by combining audio and text tweaks
Error Rate is Not Enough: Costs of Augmentation in Low-Resource ASR
Computation and Language
Summary
Automatic speech recognition (ASR) struggles when there is little training data. The authors studied how two types of audio changes and two text-based methods help ASR in low-data situations. They found that different methods fix different types of sound errors, and when combined, the two approaches work better together than alone. This combined benefit disappears with more data, so their recommendation is to pick methods based on specific speech sounds and how often they appear in training.
What this means in practice
- •For speech technology teams: Choose augmentation methods based on speech sound properties and data frequency to improve low-resource ASR accuracy.
- •For language preservation projects: Enhance ASR quality for endangered low-resource languages by mixing audio and text data augmentation techniques thoughtfully.
Authors
Katsumi Ibaraki, David Chiang
Abstract
Data augmentation is standard practice for low-resource automatic speech recognition (ASR), but its benefits are almost always reported as aggregate error rates, leaving open why and when a method helps. On four low-resource languages and English at five data sizes, we cross two augmentations that modify existing audio (speed perturbation and SpecAugment) with two that modify text transcriptions and synthesize audio from it. We analyze errors at the phoneme level and find that they help for complementary reasons. SpecAugment has the most reliable aggregate improvements, but increases tone and palatalization errors, whereas text-based methods introduce consonant insertions; in combination, each reduces the errors the other introduces. This complementarity is predictable. Tone and palatalization are each signaled by a single acoustic property and occur least frequently in training (lowest training exposure), and text-based methods that flatten word frequency raise their frequency by 37--38%, reversing those errors while further improving aggregate error. The improvement from combining the methods exceeds the sum of their individual improvements; the superadditivity vanishes by 100 hours, limiting the complementarity to the low-resource regime. Where specific error types matter, we recommend selecting augmentation by training exposure and acoustic properties, both measurable before any training.