Japanese past tense errors linked to rare verb subtype
Not All Irregularity Is Equal: Causally Isolating a Rare Failure Mode in Japanese Morphological Inflection
Computation and Language
Summary
Neural networks can usually predict Japanese verb forms correctly, but they struggle with some rare verbs that have unusual spelling and sound changes. The authors found that these rare verbs cause a lot of the mistakes, even though they appear very infrequently in training data. By removing this specific type of verb, model accuracy improves more than by removing all irregular verbs combined. This means the problem isn’t just irregular verbs, but the interaction of rare patterns and how the Japanese writing system represents these changes.
What this means in practice
- •For natural language processing teams: Improve Japanese verb inflection accuracy by targeting rare geminating verb stem errors for model refinement.
- •For language technology developers: Design training datasets and evaluation metrics that emphasize rare morphological subclasses to enhance Japanese language models.
Authors
Wen Zhang
Abstract
Neural morphological generation systems often achieve high aggregate accuracy on benchmark datasets, yet such performance can conceal systematic errors clustered in rare morphological subclasses. We present an orthography-aware diagnosis of Japanese past-tense verb inflection, treating hiragana not merely as a transcriptional medium but as a representational system that encodes morphophonological structure. Using two character-level Transformer architectures evaluated across five random seeds, we show that although both systems exceed 97% aggregate accuracy, a single structurally specific irregular subtype, verbs whose stems end in /e/ and require gemination before the past-tense suffix and make up fewer than 1% of the data, accounts for a disproportionate 30-43% share of residual errors and contributes roughly 34-48x its prevalence to total errors. We then move from diagnosis to causal isolation: controlled ablation experiments show that removing this subtype alone produces larger accuracy gains than removing all irregular verbs combined. These findings indicate that error concentration in neural morphological learning is not driven by irregularity per se, but by the interaction between extreme low-frequency morphological patterns and specific orthographic processes. We argue that morphological evaluation should incorporate fine-grained subclass analysis, and discuss implications for data-efficient, developmentally plausible language model pretraining.