Speech synthesis improves for tonal bantu language with morphology modeling
Morpho-VITS: Variational Inference with Morphological Modeling for End-to-End Speech Synthesis of a Tonal Bantu Language
Computation and Language
Summary
Text-to-speech systems struggle with Bantu languages because tones depend on complex word parts and grammar, and writing often leaves out tone information. The authors designed a new speech model that understands word structures to better guess tones. Testing on Kinyarwanda, a complicated tonal language, showed this method makes spoken words sound more natural and easier to understand. This approach helps computers speak these languages more accurately even when tone isn’t written down.
What this means in practice
- •For voice assistant developers: Create more natural voice outputs for tonal Bantu languages by incorporating morphological structure to improve tone and intonation accuracy.$Commercial implications: Enables voice assistants that speak tonal Bantu languages more naturally, a capability previously limited by tone omission in text.
- •For language preservation teams: Generate intelligible and natural synthetic speech for endangered tonal languages lacking detailed tone markings in writing.
Tested on one dataset.
Authors
Antoine Nzeyimana
Abstract
Text-to-speech models for Bantu tonal languages are challenged by a tonal system that is rooted in both the lexis (i.e., the inventory of words, stems, and affixes) and the grammar (i.e., morpho-syntax). To complicate matters, the standard writing systems of these languages often omit tone markings and syllable duration information, which must be disambiguated by the reader based on context. Motivated by linguistic descriptions of Bantu language tone systems, we propose an end-to-end text-to-speech model that augments the text encoding mechanism with a morpho-syntactic prior. We replace the standard phoneme encoder in the VITS architecture with a morpheme sequence encoder and a phoneme-to-morpheme attention network. We posit that, by using this explicit morphological modeling, we can capture the information required to produce the correct tone. Experiments conducted on the Kinyarwanda language, a tonal and morphologically complex Bantu language, reveal substantial TTS improvement from this morphological modeling. Specifically, the proposed method significantly improves the naturalness, intonation, and intelligibility of the produced synthetic voices.