Reinforcement learning improves local errors in text to speech synthesis
LoCA: Local Credit Assignment for Reinforcement Learning of Autoregressive Flow-Matching TTS
Sound
Summary
Text-to-speech systems can get feedback on how well they sound, but often this feedback is only about the whole sentence, so it's hard to fix small mistakes within it. The authors propose a method called LoCA that finds exactly where errors happen in the spoken sentence by aligning it with the written text. This helps the system focus its learning on fixing local errors. Their tests showed LoCA makes speech sound more accurate and keeps the speaker’s voice more consistent.
What this means in practice
- •For speech technology developers: Enhance TTS systems to better identify and correct local speech errors using frame-level feedback from aligned text.
- •For voice assistant engineers: Improve the quality and speaker consistency of synthesized speech in multilingual voice assistants through better local error correction.
Authors
Jie Liu, Junyu Zhou, Junhao Zhong, WeiSen Tang, Rukai Cai, Haoyue Zhan, Ruiqi Xing, Ling Long
Abstract
Reinforcement learning can improve text-to-speech synthesis through feedback on generated speech. However, sentence-level rewards do not identify local errors, while signed regression updates can lead to excessive fitting or repulsion of sampled outputs. We propose LoCA (Local Credit Assignment) for autoregressive flow-matching text-to-speech. LoCA uses ASR-based text alignment to locate content errors in generated speech and redistribute sentence-level advantages across acoustic frames. To improve training stability, we introduce reference-relative positive and negative thresholds that limit continued fitting and repulsion. Experiments on VoxCPM2 show that LoCA reduces sample-weighted content error on a multilingual CV3 evaluation subset by 20.5% and 14.3% relative to the pretrained backbone and GROW, respectively, while improving speaker similarity. LoCA also improves content accuracy on the Seed-TTS-Eval hard subset. Ablation results support the contributions of local credit and threshold-based regression control.