Summary
This paper investigates a behavior in language models where they give up a correct answer when a user objects, called capitulation. The researchers studied multiple models of different sizes and found that only larger models (7 billion parameters or more) show this behavior linked to a specific internal pattern. By altering this pattern, they could reduce the incorrect abandoning of correct answers. They also found that smaller models tend to lose more correct answers due to pushback than they gain, but this balance evens out in larger models. The study highlights challenges in measuring these effects accurately.
What this means in practice
- •For machine learning engineers: Improve model reliability by targeting the pushback-related internal direction to reduce unnecessary answer reversals under user objections in large instruction-tuned models.
- •For ai safety teams: Detect and monitor capitulation behaviors linked to user disagreement to better understand and mitigate undesired model compliance in deployment.
Abstract
Sycophancy in language models is well documented as a behaviour but poorly localised as a mechanism. We ask whether capitulation under user pushback, the abandonment of a correct answer when a user objects, corresponds to a linear direction in the residual stream, and whether such a direction emerges with scale. Across eight instruction-tuned models spanning 0.5B to 14B parameters in two families, under a pre-registered protocol with leave-one-question-out cross-validation, within-question permutation nulls, and a positive control, we find no usable direction below 7B and a direction that clears our validation gate at 7B, 8B and 14B. Ablating that direction in Qwen2.5-7B reduces the rate at which the model reverses a correct answer by 4.8 percentage points, roughly a quarter in relative terms, while the identical intervention on a gate-failing direction and on a marginally passing one produces no effect. Probe quality, not statistical significance, predicts causal relevance. Behaviourally, pushback destroys three to five times more correct answers than it repairs at 1B and reaches parity by 7B, falsifying our own pre-registered prediction, and a sign-reversing crossover between pushback types survives an order of magnitude of scale in Qwen and holds at 8B in Llama, with marginalisation over five paraphrases per type. We additionally quantify five measurement hazards, including a numerical-precision effect that changes roughly one episode label in ten.