Parser agreement and disagreement in L2 Korean UD: Implications for human-in-the-loop annotation
2026-05-07 • Computation and Language
Computation and Language
AI summaryⓘ
The authors created an easier way for people to check Korean language data by using two computer programs that analyze sentences. They tested if the programs agreeing with each other means the results are likely right, and found that it often does match what humans think. They also noticed the programs disagree in certain tricky grammar parts, which helps understand where machines need improvement. Some disagreements are fixable, but others show hard problems in analyzing Korean sentences.
morphosyntactic annotationsecond language (L2)Koreandependency parsingparser agreementuniversal dependencies (UD)grammatical relationsclause boundariessemi-automatic annotationdomain adaptation
Authors
Hakyung Sung, Gyu-Ho Shin
Abstract
We propose a simplified human-in-the-loop workflow for second language (L2) Korean morphosyntactic annotation by leveraging agreement between two domain-adapted parsers. We first evaluate whether parser agreement can serve as a proxy for annotation correctness by comparing it with independent human judgments. The results show strong correspondence between parser and human judgments, supporting the feasibility of semi-automatic L2-Korean UD annotation. Further analysis demonstrates that parser disagreements cluster in linguistically predictable domains such as grammatical-relation distinctions and clause-boundary ambiguity. While many disagreement cases are tractable for iterative model refinement, others reflect deeper representational challenges inherent in parsing and tagging L2-Korean corpora.