Self-distilled method improves rare word and accent control in text-to-speech
Self-Distilled Pronunciation and Accent Control for Neural Text-to-Speech
Sound
Summary
Text-to-speech systems often struggle to pronounce uncommon words correctly or with the right accent because they typically guess from spelling. The authors developed a technique where the system learns to improve its own pronunciation by using its correct reading of common words as a teacher for rare or accented words. This method works well without needing extra recorded speech or word-by-word corrections and does not reduce the natural sound of the speech. The approach also transfers well to different types of text-to-speech models with some exceptions.
What this means in practice
- •For voice assistant developers: Enhance rare word pronunciation and accent accuracy in speech outputs without collecting extra labeled speech data.
- •For audiobook production teams: Improve natural-sounding speech synthesis with better accent control on uncommon words to match speaker style using existing text-to-speech backbones.
Authors
Shuhei Kato
Abstract
Text-to-speech that reads raw text has no lexicon: a rare word is read as guessed. Remedies train a reading-and-accent channel on recorded speech or edit words one at a time from exemplars. We do neither. The frozen backbone reads a sentence containing a common word it already says correctly, and its own output then serves as the teacher for the same sentence, with that word replaced by a tagged, accented reading; this training pair is the whole idea. On Sarashina2.2-TTS, screened raters at Fleiss' kappa = 0.85 hear the prescribed accent on 0.89 of unseen words against 0.57 for kana, which cannot express one; kana wins no pair; naturalness is not measurably hurt. Moved untuned to autoregressive, diffusion, and encoder-decoder backbones, it transfers reading, 0.25 to 0.47 above no edit on 319 words, and on CosyVoice 2 accent on two words in three, but not on Irodori; the paper locates why.