Separating feature steering from use in large language models
Does the Model Use the Feature? Separating Steering from Mechanism in LLMs
Artificial Intelligence
Summary
Large language models (LLMs) can have features inside them that track certain ideas or concepts. Sometimes changing these features changes how the model behaves, but this doesn't always mean the model really uses that feature the same way it does naturally. The authors developed tests to see if a feature is truly part of the model’s internal workings or just something that can be tweaked to change behavior. Their tests show that just because you can steer a model’s behavior by changing a feature doesn’t prove the model relies on that feature when working normally. The meaning depends on how the test is done and which feature is studied.
What this means in practice
- •For llm developers: Check if manipulating internal features in your models truly reflects how the model uses those features naturally.
- •For ai safety engineers: Use tests to verify whether steering model behavior affects genuine internal computations or only superficial changes.
Authors
Tong Che, Yilong Li
Abstract
Internal features in LLMs are often interpreted as mechanisms when they track a concept and their manipulation changes a related behavior. Yet steering can push a feature far outside its natural range, where its effects need not reflect the model's own computation. We examine this inference and propose an empirical contract whose tests evaluate features at values observed on natural inputs. One test copies a feature's value from an input that shows a behavior into a matched input that does not (installation) or the reverse (removal); the other restores the feature after an upstream edit (downstream rescue). Installation measures how far the feature suffices for the behavior; removal and downstream rescue measure how much the model uses it. Applied to three kinds of representations, the two strengths separate sharply. The published unknown-entity latent strongly steers knowledge abstention, yet installing observed values from either published latent into matched prompts transfers only a small fraction of the natural known--unknown abstention contrast. Dense known--unknown directions show opposite asymmetries between installation and removal in Gemma and Llama, and how fully a released subject--verb agreement feature set reproduces and restores the behavior depends on how its values are written into the model. Tracking a concept and steering a behavior therefore do not by themselves show that the model uses a feature, and each conclusion holds only for the intervention tested.