Yo-ByT5 improves Yoruba text restoration with fewer model parameters
Yo-ByT5: Efficient and High-Fidelity Diacritic Restoration for Yorùbá
Computation and Language
Summary
Yorùbá is a language where little marks called diacritics are needed to understand words correctly. But many Yorùbá texts are written without these marks, making it hard for computer programs to understand them. The authors created a new computer model, Yo-ByT5, to add these marks back into the text. Their model works as well as the best ones currently available but uses fewer resources. They also shared their tools and suggest making a bigger test set for this problem.
What this means in practice
- •For natural language processing teams: Improve Yoruba text inputs by automatically restoring missing diacritics with efficient models that reduce resource needs.
- •For machine translation developers: Increase Yoruba translation accuracy by using diacritic restoration as a preprocessing step to clarify word meaning.
Authors
Ahmad Samuel Gali, Shamsuddeen Hassan Muhammad
Abstract
Yorùbá is a widely spoken tonal language that depends on diacritics to avoid lexical ambiguity. However, it is often written without these diacritics, thereby hindering downstream Natural Language Processing (NLP) tasks. In this paper, we introduce Yo-ByT5, a byte-level Automatic Diacritic Restoration (ADR) model fine-tuned from ByT5-small. We evaluate Yo-ByT5 alongside five publicly released Yorùbá ADR models and one open-weight large language model (LLM) on the YAD benchmark under a consistent protocol. Our results demonstrate that Yo-ByT5 matches the performance of the strongest existing model, mT5-base, with a DER of 10.14% and a CER of 3.48%. Furthermore, it exhibits superior text fidelity despite using approximately half the parameter count of mT5-base. We also release our training code and model outputs, as well as call for the development of a larger, purpose-built benchmark for Yorùbá diacritic restoration.