Controlling Refusal Behavior of LLMs via Stiefel-Constrained Rotation Steering
2026-08-31 • Machine Learning
Machine LearningComputation and Language
AI summaryⓘ
The authors present a new way to control when large language models refuse to answer by changing how the model's internal signals (activations) rotate, without needing extra parts like refusal vectors. They use a math technique called Riemannian optimization to learn these rotations efficiently. Their tests show this method works better than earlier ones, and they explore which ideas in their method matter most. Overall, the authors suggest their rotation approach is a good step toward reliably steering language model behavior.
Activation steeringLarge language modelsRiemannian optimizationRotational transformationsModel refusalIntervention efficiencyAblation studyRefusal vectorsGeometric methodsParameter efficiency
Authors
Kirill Bunin, Dmitry Bylinkin, Vladimir Aletov, Daniil Medyakov, Vladimir Solodkin, Aleksandr Beznosikov
Abstract
Activation steering has emerged as a lightweight approach for controlling model refusal at inference time. A growing line of research explores trainable rotations of activations to develop geometrically principled intervention mechanisms. However, existing techniques rely on auxiliary constructs, such as refusal vectors, to define these rotations. In our work, we develop a self-contained methodology for learning parameter-efficient rotational transformations based on Riemannian optimization. We empirically validate the proposed scheme, demonstrating its superiority in intervention efficiency. An extensive ablation study highlights the importance of key design choices in our method. Our results identify the proposed rotation-based steering scheme as a promising direction for more reliable control over the behavior of LLMs.