Phonologically informed tokenization aids German speech recognition on diverse speech types
Phonologically Informed Tokenization for German Speech Recognition: A Cross-Domain Study
Computation and LanguageMachine Learning
Summary
German speech recognition is challenging because words can be complex and change depending on dialect or speaking style. The authors compared three ways to break down speech into parts for a computer system to understand. They found that using phonology-based parts, like syllables informed by sound patterns, works as well as other popular methods for clear read speech and even better on dialects and casual speech. This suggests how speech is split into smaller units affects recognition depending on vocabulary size and speech style, rather than just the type of linguistic information.
What this means in practice
- •For speech recognition engineers: Choose tokenization methods based on expected vocabulary size and speech style shifts to improve German ASR performance across dialects and spontaneous speech.
- •For language technology developers: Integrate phonologically informed tokenization using syllable structure in speech systems to better handle varied German speech including dialects and spontaneous utterances.
Authors
Christopher Witzl, Tobias Bocklet, Korbinian Riedhammer
Abstract
German is a morphologically rich language whose syllable structure is exceptionally well-predicted by the Knuth--Liang hyphenation algorithm. We ask whether phonologically informed tokenization can serve as a competitive target for end-to-end speech recognition. We compare three tokenizer families on the Omnilingual ASR wav2vec 2.0 backbone fine-tuned with CTC: the pretrained multilingual character inventory, a data-driven Byte-Pair Encoding (BPE) over orthography, and phonologically informed units from Pyphen syllabification and grapheme-to-phoneme conversion. Across 40 fine-tunes, we evaluate on three German test sets spanning orthogonal shifts: in-domain read speech, dialectal spontaneous speech, and standard-German spontaneous speech. In-domain, all phonologically informed tokenizers match BPE and the multilingual character baseline on both WER and CER. Under domain shift the picture splits along vocabulary size rather than the linguistic axis of variation: at small vocabularies, syllable-aware tokenization improves on dialectal speech, where phonetic surface forms vary but syllable structure is preserved, and stays ahead on spontaneous speech, where new word-forms violate vocabulary closure. A phoneme-level confusion analysis further shows that all tokenizers commit the same canonical function-word errors, indicating that the acoustic encoder, not the tokenizer, dominates the error topology. Our findings suggest that tokenizer choice may depend on the vocabulary budget as much as on the distribution shift expected at deployment rather than reducing to a single universal optimum.