Large language models struggle with Korean new words but infer meanings
KoAGE: Do Large Language Models Know, Learn, and Coin Korean Neologisms?
Computation and LanguageArtificial Intelligence
Summary
New words in Korean come from many different ways, making it hard for language AI to understand them fully. The authors made a new tool called NeoTrace that tracks and dates new Korean words so AI can be tested better. They tested 21 language models and found they have a hard time knowing new words by name but can guess their meanings from how they are used. However, AI still isn’t good at understanding how these words are formed. Sometimes AI can create new words similar to humans, especially when working together.
What this means in practice
- •For natural language processing developers: Improve Korean language models by using KoAGE to diagnose knowledge gaps about new words and test their ability to infer meanings from context.
- •For ai product teams: Design AI systems that better handle newly coined Korean words by integrating multi-agent coinage insights to generate plausible new vocabulary.
Authors
SungHo Kim, Eojin Jeon, SangKeun Lee
Abstract
Korean neologisms arise through diverse mechanisms, a demanding test of language models beyond lexical memorization. To enable systematic evaluation, we introduce NeoTrace, an automated, rerunnable framework that collects and dates neologisms across genres. Its 4,271 entries support KoAGE, a cutoff-aware benchmark with three acquisition and two generation tasks---the latter, to our knowledge, the first to compare model coinages with attested neologisms under their original materials and community context. Across 21 models, lexical recall diverges from contextual inference: in a five-model temporal analysis, models define only 13-30% of cutoff-year words, yet can infer meaning from one usage. However, structural understanding remains limited: even correct definitions barely improve formation analysis of pre-cutoff neologisms. Given word-building materials, single models often follow human formation mechanisms, whereas multi-agent coinage favors abbreviation and depends on interaction framing. KoAGE thus separates what models know, learn from use, and coin.