Language models improve word meaning recognition with sense clues
WiC is Not WSD: A Study on LLMs and Lexical Ambiguity Resolution
Computation and Language
Summary
Understanding the exact meaning of a word in different sentences is hard for language models. The authors studied how large language models handle this task and found that giving them a list of possible meanings helps them do better. They also showed that some mistakes come from unclear boundaries in meaning rather than model confusion. The study reveals that models often make errors by thinking about too many tiny differences in word meanings.
What this means in practice
- •For natural language processing engineers: Improve word meaning detection in language tasks by integrating explicit sense candidates into model inputs for more accurate context judgments.
- •For content moderation teams: Use models enhanced with sense information to better interpret ambiguous words in user content and reduce misclassification errors.
Authors
Yi Zhou, Kiamehr Rezaee, Danushka Bollegala, Mohammad Taher Pilehvar, Jose Camacho-Collados
Abstract
Word-in-Context (WiC) remains challenging for language models, despite recent progress on lexical-semantic tasks. We hypothesise that this difficulty arises not only from comparing two contextual uses of a word, but also from the absence of an explicit sense inventory that specifies the relevant level of semantic granularity. We evaluate open LLMs on WiC and traditional Word Sense Disambiguation (WSD) under similar settings. We find that providing candidate senses, similar to what is done in traditional WSD, improves WiC performance in all settings. In general, explicit sense information helps models make more consistent and targeted judgements. Human evaluation further shows that many apparent WiC errors reflect label ambiguity or mismatches between model and annotator sense boundaries rather than simple failures of lexical understanding. In particular, results show that LLMs overthink the sense distinction often leading to errors based on overly fine-grained distinctions.