Quantifying and Mitigating Korean Jamo-Level Typographical Vulnerabilities in Large Language Models
2026-08-31 • Computation and Language
Computation and Language
AI summaryⓘ
The authors study a special type of typo in Korean, where mistakes can happen within a syllable because Korean characters are made of smaller parts called jamo. These typos confuse language models and existing tools, making their answers less accurate. They tested several models and found that more typos lead to worse performance, and bigger models are not better at handling these errors. The authors also created a way to detect these typos inside the model and use this signal to decide when to use a more careful reasoning method, improving accuracy without much extra computing cost.
Korean jamotypographical perturbationcharacter-level editssub-word tokenizationgrammatical error correctionlanguage modelschain-of-thought reasoninglinear probesaccuracy degradation
Authors
Seojin Lee, Hwanhee Lee
Abstract
Korean introduces an additional typographical perturbation level not captured by ordinary character-level edit models: because syllable blocks are internally composed of sub-character units called jamo, keyboard-level errors can occur within a syllable, either producing a valid but semantically altered character or exposing raw jamo on the surface. Both outcomes disrupt sub-word tokenization and are not reliably corrected by existing grammatical error correction pipelines, leaving LLMs directly exposed to corrupted inputs. To quantify this vulnerability, we apply five jamo-level perturbation types to the KMMLU benchmark and evaluate four language models, finding that accuracy declines monotonically with perturbation intensity and that parameter scaling does not confer robustness against intra-syllabic noise. We further show that typo-corrupted inputs induce a distinct shift in internal representations that is not reducible to ordinary answer incorrectness, and that a simple linear probe trained on these representations detects unseen perturbation types with high AUROC. Motivated by this signal, we propose Typo-Aware Chain-of-Thought (TACoT), which routes inputs to chain-of-thought inference only when the probe detects a likely typo, recovering a substantial portion of the CoT accuracy gain at a fraction of the inference cost.