Korean text detection improves with linguistics based robust methods
Beyond Surface Cues: Robust LLM-Generated Korean Text Detection
Computation and Language
Summary
Detecting text written by large language models (LLMs) is important but most methods focus on English, ignoring Korean. The authors created a challenging Korean text dataset called KoLing that avoids easy tricks for detection, making tests more reliable. They also designed KoLingProbe, a detector that uses Korean grammar and language features to spot generated text more accurately. Their approach works well even against tricky attempts to fool detectors and explains its decisions better than previous methods.
What this means in practice
- •For content moderation teams: Improve detection of AI-generated Korean text in online platforms to reduce misinformation and enforce content policies.
- •For language technology developers: Develop tools that detect and analyze AI-generated Korean text using linguistics-based robust features for better reliability.
Authors
SungHo Kim, SangKeun Lee
Abstract
As large language models (LLMs) generate increasingly fluent text, the need for reliable detection of LLM-generated content has grown substantially. However, existing research remains heavily English-centric, leaving Korean largely underexplored. Moreover, current evaluations of Korean text detection often rely on superficial cues, leading to overly optimistic estimates of detection performance. To address these limitations, we introduce \textsc{KoLing}, a bias-controlled adversarial benchmark comprising 50.9k instances that suppresses superficial shortcuts while preserving semantic content and separately evaluates LLM-generated and LLM-modified text detection. We further propose KoLingProbe, an interpretable detector grounded in theoretical linguistics that models Korean-specific properties across multiple linguistic levels. Experiments show that KoLingProbe substantially outperforms strong baselines on all detection tasks while remaining robust to adversarial attacks. Our analyses further reveal that integrating complementary linguistic features across multiple levels yields both improved performance and interpretability.