Language models reveal feared words that trigger refusals
Which Words a Model Fears: From a Refusal Direction to a Vocabulary-Wide Fear Lexicon
Computation and Language
Summary
Language models sometimes refuse to answer based on specific words in a prompt, even if those words don’t change its meaning. The authors found a hidden direction in the model’s internal state that measures how 'feared' a word is—words along this direction make the model more or less likely to refuse. This lets them create a full vocabulary ranking of words from ‘feared’ to ‘courageous.’ Changing feared words to courageous synonyms in prompts helps models give more answers instead of refusing.
What this means in practice
- •For ai safety teams: Improve refusal behavior by identifying and replacing feared words to reduce unnecessary refusals in language model outputs.
- •For chatbot developers: Use the fear lexicon to adjust prompt wording dynamically and increase response rates by avoiding words that trigger refusals.
Authors
Boyuan Zhang, Mohammad Zandsalimy, Shanu Sushmita
Abstract
Whether an aligned language model refuses depends in part on which words a prompt contains, regardless of whether those words carry the request's meaning. Harm words shift the hidden state at the generation boundary along a component shared across prompts and words, which crossed differencing extracts as a single direction. In eight models, writing along this direction raises and lowers refusal monotonically with dose, ahead of random and KL-matched directions. Used as a ruler over the full vocabulary, the direction gives every word a shift, from "feared" words at one end to "courage" words at the other; a word's shift is predictable from its own representation, and appending a feared word to a benign prompt is enough to make the model refuse it. Two held-out tests validate the ranking in all eight models: refusal after inserting an unseen word rises with its table shift, and replacing a prompt's most feared word by its most courageous meaning-preserving synonym significantly raises the answer rate on benign and harmful prompts alike, by an amount that follows the shift the replacement actually causes.