AI summaryⓘ
The authors study why language models sometimes give answers that seem overly confident or influenced by social pressure, even when their actual internal beliefs haven't changed. They propose a new method to ensure models only change their answers based on real evidence, not external pressures, by focusing on parts of the model's output that can be independently controlled. Their technique uses interventions to test and certify that the model's reported confidence is stable under these changes. They demonstrate this approach on benchmarks designed to simulate trustworthy vs. pressured situations and show it works across different models. This work provides a way to better understand and control how models report their certainty without changing the underlying beliefs.
aligned language modelsincentive-compatibilitycounterfactual report mediatorsBayesian witness benchmarkcausal interventionsconfidence calibrationreport coordinatessycophancycounterfactual invarianceactivation-level analysis
Abstract
Aligned language models routinely misreport under non-evidential incentive pressure: they agree with a confident user or overstate certainty even when their internal belief is unchanged. We cast this as a failure of internal incentive-compatibility (IC) and present a method for learning and certifying counterfactual report mediators that hold a model's reports to a causal contract: invariant to forbidden influences (pressure, prestige, restyling) and responsive to licensed ones (genuine evidence). These two demands, resist and update, pull in opposite directions. We study them on a Bayesian-witness benchmark with known posteriors, in which the same user disagreement is licensed evidence or forbidden pressure purely by stated source reliability. We (i) causally identify, by interchange interventions rather than probe accuracy, low-rank report coordinates for answer, confidence, and caveat that are near-orthogonal and independently controllable, and (ii) introduce a training-free counterfactual report-coordinate (CRC) clamp that references the model's own report under a counterfactually incentive-neutralized context. On the witness benchmark the two-pass clamp attains resist and update of 1.00 jointly (Wilson 95% CI [0.99,1.00]), a causal certificate under a constructible reference, not a deployed solution. Global decoding and steering show a single-parameter tradeoff; output-level fine-tuning matches both objectives only when both are enumerated; resist-only training loses evidence-responsiveness. The deployable single-pass compilation is lossy (0.73/0.97). The mechanism and clamp reproduce across three model families and transfer to a natural sycophancy benchmark (SycophancyEval). Our contribution is the interface and certification method: activation-level counterfactual incentive-invariance as a structural primitive for internal IC.