Gaussian process optimization improves word hyphenation patterns efficiently
The Art of Hierarchical Competing Patterns: Gaussian Process Optimization of Hyphenation
Computation and LanguageDigital LibrariesMachine Learning
Summary
Breaking words correctly at the end of a line is important for neat text layouts, and this uses rules called hyphenation patterns. These rules usually need expert tuning, which can be slow and inconsistent. The authors show that a machine learning method called Gaussian-process Bayesian optimization can automatically find better rules that both improve accuracy and reduce rule size. They tested this on many languages and found it usually beats expert-made rules, making the process easier and more reliable.
What this means in practice
- •For software developers for text processors: Automatically generate compact and accurate hyphenation pattern rules for text rendering engines in multilingual software.
- •For web browser developers: Improve word wrapping quality and performance in web browsers by optimizing hyphenation pattern parameters without manual trial-and-error.
Authors
Ondřej Sojka, Petr Sojka
Abstract
Hyphenation patterns remain a compact and widely deployed solution for word breaking in typesetting systems, text processors, and web rendering engines, but their generation still depends on manually tuned patgen program parameter profiles. We formulate patgen profile selection as a black-box hyperparameter optimization problem and evaluate Gaussian-process Bayesian optimization for this task. The search objective combines a precision-oriented F_{1/7}-score with an explicit trie size-accuracy trade-off using a normalized trie-size penalty. We evaluate the method on 17 hyphenated word-list datasets covering 14 languages and multiple scripts. Against two strong hand-tuned profiles regenerated from the same 8/10 training split and evaluated on the same 1/10 held-out test split, the GP-optimized profiles improve F_{1/7} on 16 of 17 datasets and reduce trie size on all 17. The median optimized/baseline trie ratio is 0.407. A dataset-level sign test gives p = 1.37e-4; a separate budget-matched comparison on five representative datasets shows that systematic search is competitive and usually improves over the best hand-tuned profile under the fixed comparison objective. The results show that model-based optimization can make pattern generation more reproducible and less dependent on expert trial-and-error while keeping the accuracy-compactness trade-off explicit.