Language models adjust word meanings based on context clues
Lexical Redefinition Reveals Context-Sensitive Instruction Following
Computation and Language
Summary
Sometimes a language model is told to think of a word in a new way, but the sentences that come after can change how well the model follows that instruction. The authors tested over 800 word pairs and found that the model doesn't always treat the instruction as a strict rule. Instead, it listens to the next sentence and might keep, ignore, or even do the opposite of the instruction depending on the new context. This shows language models mix instructions with new information instead of just following rules blindly.
What this means in practice
- •For ai developers: Improve language model prompt designs by accounting for how follow-up sentences affect word interpretation instructions.
- •For chatbot builders: Create chatbots that better handle ambiguous instructions by integrating conflicting language cues within conversations.
Authors
Roksana Goworek, Haim Dubossarsky
Abstract
How do language models reconcile conflicting contextual evidence within a prompt? We study this question using lexical redefinition, where an explicit instruction specifies how a word should be interpreted and a subsequent sentence provides additional evidence through its usage. Across 870 word pairs, we find that lexical redefinitions are not applied as fixed rules. Instead, the behaviour depends systematically on the relationship between the explicit instruction and the intervening sentence: the contextualised usage can preserve, weaken, or completely reverse the instruction. These results suggest that the interpretation of an explicit instruction remains sensitive to subsequent contextual evidence. Lexical redefinition therefore provides a controlled behavioural paradigm for studying how language models integrate multiple sources of information within a prompt.