Language models resist pressure and use evidence more accurately
Suppressing Pressure, Amplifying Evidence: Self-Guided Attention Steering to Mitigate Sycophancy and Stubbornness
Artificial IntelligenceComputation and Language
Summary
Language models can sometimes wrongly agree with a user's incorrect opinion or stubbornly ignore new information that should change their answer. The authors created a way to test if fixing one problem makes the other worse and designed a method called SPAE that guides the model's attention to trust evidence over pressure. This approach helps the model update answers correctly and avoid giving false agreement to users. Their tests show SPAE improves performance across multiple language model types without extra training.
What this means in practice
- •For chatbot developers: Improve chatbot responses to resist misleading user claims and better incorporate factual context during conversations.
- •For customer service platforms: Make virtual assistants less prone to falsely agreeing with customers and more reliable when adjusting answers based on new information.
Authors
Yinghao He, Mengyu Xu, Haixiang Sun, Donghan Li, Yibo Wang, Lixu Wang, Kezhen Chen, Chi Li, Chunwei Liu, Bharat Bhargava, Chongyang Gao
Abstract
Reliable language models should resist unsupported user pressure while effectively using objective contextual information. However, models may exhibit sycophancy by yielding to unsupported user pressure or contextual stubbornness by failing to update their answers when relevant contextual information warrants revision. Evaluating interventions for these failures separately can obscure whether mitigating one failure exacerbates the other. To assess this trade-off, we introduce CoPE-Bench with six conditions per question: a neutral baseline, correct or incorrect user pressure, contextual information consistent with or conflicting with the neutral answer, and a joint condition combining incorrect claims with conflicting contextual information. To regulate the influence of user pressure and contextual information, we propose SPAE (Suppressing Pressure, Amplifying Evidence), a training-free framework that uses the model's own judgments to identify relevant tokens, suppressing user pressure and amplifying contextual information through token-level attention steering. On average across five backbones, SPAE reduces pressure following by 18.8 percentage points and increases joint-condition updating by 5.5 percentage points relative to the strongest baseline in the main comparison. In two-turn dialogue, it improves joint-condition updating by an average of 13.2 percentage points over the strongest prompting baseline. The source data and codes can be found at https://github.com/03Grant/sycophancy-and-stubbornness.