Repetition causes counting errors in neural text-to-speech models
Repetition, Not Length: Isolating the Counting Failure in Neural Text-to-Speech
Computation and LanguageSound
Summary
Text-to-speech systems sometimes make mistakes when reading aloud repeated phrases many times, not because the text is long but because of the repetition itself. The authors tested sentences with repeated words and compared them to sentences of the same length but without consecutive repeats. They found that six different text-to-speech models performed well on the non-repetitive sentences but failed badly on those with repeated words. This problem persists even with different decoding methods and recognition tools, and it worsens as the pattern of repetition becomes more frequent.
What this means in practice
- •For text-to-speech developers: Improve text-to-speech software by focusing on handling repeated phrases to reduce counting errors and improve speech quality for repetitive content.
- •For voice assistant engineers: Design voice assistants that better handle repeated commands or phrases by addressing the counting failure revealed in neural TTS systems.
Authors
Kirill Borodin, Vasilii Kudryavtsev, Maxim Maslov, Grach Mkrtchian
Abstract
Text-to-speech models loop, truncate and lose count on text that repeats a phrase many times. We show that repetition itself is what breaks them, not the length that comes with it. Every repeated sentence in our test set is paired with a control of matched sentence and word count in which no word ever repeats back-to-back. Six models from three architectures render the controls almost perfectly and fail the repeated twins: 94.3% against 18.2% exactly right at k >= 6. The gap survives greedy decoding, repetition-penalty sweeps, four independent speech recognisers and 420 analysis specifications without once reversing sign; a held-out fourth architecture lands within a point of its predicted gap, and one of two non-autoregressive baselines shows the same failure. Varying the period of the text shows the failure grows smoothly with periodicity, half of it surviving when no word is adjacent to itself.