Affixal negations improve language model understanding more than common words
Not all Negation Cues are Equal: Affixal Negations Yield Better Negation Understanding
Computation and LanguageArtificial Intelligence
Summary
Understanding when a sentence says something is not true is tricky for language computers. This work shows that not all ways of saying 'not' are equally helpful for these models. The researchers created a huge collection of sentences with many different types of negations, including tricky ones made by adding prefixes or suffixes to words. They found that training computers on these affixal negations helps them understand negation better than just using common negation words like 'not' or 'never'.
What this means in practice
- •For natural language processing engineers: Improve language model training by including affixal negations to boost negation understanding in text applications.
- •For chatbot developers: Enhance chatbot responses by refining negation comprehension through pre-training on diverse negation types beyond simple words.
Authors
Tian Tan, Eduardo Blanco
Abstract
Negation remains a longstanding challenge for both language models (LMs) and large language models (LLMs). Prior work mainly focuses on a small set of high-frequency single-word negation cues, such as not and never, with limited exploration of broader negation types and modern LLMs. To address this gap, we construct NegCue, a large-scale dataset containing over 1.8M samples spanning single-word, multi-word, and affixal negation with more than 200 unique cues. We further pre-train both encoder-only LMs and LLMs on NegCue to investigate how different negation types affect negation understanding. Experiments on five downstream benchmarks show that negation types contribute unevenly to performance gains under the same training scale. In particular, affixal negation yields the largest improvements, while the gains from the commonly studied single-word negation remain modest. Moreover, our results demonstrate that further pre-training improves negation understanding for both LMs and LLMs.