Model defaults control how activation steering affects language output
Steering Interference Reflects the Model's Defaults, Not the Behavior Directions
Machine LearningArtificial Intelligence
Summary
Activation steering tries to change how language models behave by nudging certain directions in their internal activations. The authors find that instead of making models perform new behaviors cleanly, these nudges mostly pull the model toward a small set of usual behaviors it prefers, like refusal or poetic language. This effect is largely determined by the model itself, not the specific behavior being targeted. They observed this pattern across many behaviors and models, suggesting that simply isolating behavior directions won’t achieve perfect modular control.
What this means in practice
- •For language model developers: Improve understanding of why behavior control via activation steering has limits due to model-internal biases, guiding better steering methods.
- •For ai safety engineers: Assess risks of unexpected behavior shifts when applying steering techniques to language models, informing safer deployment strategies.
Authors
Srikanth Malla, Chiho Choi, Joon Hee Choi
Abstract
Activation steering promises modular control of language model behavior: a behavior such as politeness corresponds to a direction in a model's activations, and adding that direction while it generates should switch the behavior on and leave everything else alone. It does not. We ask what decides which other behaviors move, and by how much, and find that it is the model rather than the behavior being steered. A steer relaxes the model toward a small set of behaviors it already favors, chiefly refusal, sycophancy, and poeticism, and that set is much the same whatever is steered. Three results across 24 behaviors and ten instruction-tuned models support this, every effect read off the generated text by a language-model judge rather than off a probe. That readout matters: all 24 behaviors are linearly decodable, but only 20 change what the model writes. First, a direction carrying no behavioral content, matched to a real steer only in the size of the vector it adds, moves the same behaviors in the same order as real steers do, while producing none of the behaviors that need a specific direction. Second, most interference runs one way, so it cannot be an overlap between two directions: steering profanity makes the model toxic, while steering toxicity leaves profanity untouched. Third, with a behavior held out entirely, geometry measured on the others explains almost none of the interference it takes part in. The account holds on all ten models, the pull toward defaults strongest below 10B parameters and weakening in each family's largest. Reading a steer as a perturbation whose endpoint the model fixes implies that disentangling behavior directions cannot by itself make steering modular.