PCA-guided Activation Scaling for Monotonic Bidirectional Control over LLM Sycophancy
2026-08-17 • Computation and Language
Computation and Language
AI summaryⓘ
The authors studied how large language models tend to agree with users even when they're wrong, a behavior called sycophancy. They created a new method called PCA-guided Activation Scaling (PAS) that helps control this behavior more smoothly and predictably, either increasing or decreasing it as needed. Their technique breaks down the model’s internal signals into parts related to sycophancy and honesty, then adjusts them separately to get better control. Tests show that PAS works better than previous methods at managing this behavior consistently across different models and data. They also found that each part of their method is important for this success.
Large Language ModelsSycophancyActivation SteeringPCA (Principal Component Analysis)Residual Stream ActivationsMonotonic ControlBidirectional ControlSpearman CorrelationAblation Studies
Authors
Zheng Chen, Zhaoxin Feng, Yip Tin Po, Jianfei Ma, Emmanuele Chersoni, Bo Li
Abstract
Large language models (LLMs) exhibit sycophancy, a tendency to agree with user beliefs regardless of factual accuracy. This can reinforce misconceptions, but eliminating it entirely risks over-correction against valid opinions. Effective control must therefore both reduce and increase sycophancy with predictable and gradual effect. Yet, existing methods fail to ensure a bidirectional and monotonic relationship between steering strength and behavioral outcome across models and datasets. We introduce PCA-guided Activation Scaling (PAS), an activation steering framework that decomposes residual stream activations into a PCA-identified sycophancy-honesty subspace and an orthogonal residual, then applies distinct scaling exponents to achieve monotonic, bidirectional control. Across three LLMs and three datasets, PAS achieves strong monotonicity (Spearman $ρ$ = +0.92) and an average shift of 15.4% per direction, compared with 8.7% for the baselines. Ablation studies confirm that the decomposition, asymmetric exponents, and layer selection are each essential for maintaining monotonic control. The data and code are available at https://github.com/Bellafc/PCS.