Khmer text normalization toolkit improves speech technology accuracy
THA: Weighted Finite-State Text Normalization and Inverse Text Normalization for Khmer
Computation and Language
Summary
Converting written Khmer into the way people speak and back again is hard because the script doesn’t use spaces between words and mixes numbers into words. The authors developed Tha, a computer program that breaks lines of Khmer text into words and changes them into spoken forms correctly using a special mathematical method. When tested, this tool correctly handled most examples, making it easier to build speech recognition and text-to-speech systems for Khmer. Tha is freely available for anyone to use or improve.
What this means in practice
- •For speech technology developers: Improve Khmer speech systems by reliably converting text to spoken form and back without error-prone manual rules.
- •For natural language processing engineers: Build tools that segment and normalize Khmer text in one step using weighted finite-state methods for accurate downstream processing.
Authors
Seanghay Yath
Abstract
Text-to-speech needs written text in spoken form, and speech recognition output needs the reverse. For Khmer, neither direction has a maintained open-source tool, and the script makes both harder: words are not separated by spaces, and number words occur inside ordinary words. We present Tha, a Khmer text normalization and inverse text normalization toolkit built from weighted finite-state transducers. It segments and classifies a whole line in one shortest-path search, and a second transducer rejects token boundaries inside a Khmer syllable. On Google's Khmer test suite, Tha agrees with the reference on all 274 cardinals up to one spelling variant, and on 2,906 real TTS prompts, 153 of the 158 sentences it rewrites are correct. Tha is open source under the Apache 2.0 license.