Fast Multi-dimensional Refusal Subspaces via RFM-AGOP
2026-07-02 • Artificial Intelligence
Artificial IntelligenceMachine Learning
AI summaryⓘ
The authors studied ways to find and control groups of behaviors inside large language models, especially when the model refuses to answer harmful questions. They improved a method called Recursive Feature Machine (RFM) to quickly identify these multi-dimensional behavior patterns in different models. Their version of RFM works faster and better than older methods, even on models that generate long reasoning steps. They plan to explore how these behavior groups relate to ones found by other methods. If successful, their approach could provide a cheaper, scalable way to analyze and steer language models.
Large Language ModelsBehavior SteeringRefusal SubspaceRecursive Feature MachineModel InterpretabilityMulti-dimensional SubspacesAblation TaskReasoning TracesProbe InitializationComputational Efficiency
Authors
Thomas Winninger
Abstract
Steering and monitoring activations in Large Language Models (LLMs) are increasingly used for both safety and interpretability. Early work assumed behaviours are encoded along single linear directions, but recent findings suggest complex behaviours, such as the refusal to answer harmful queries, live in multi-dimensional subspaces. However, existing methods for extracting these subspaces are computationally expensive, which becomes prohibitive on reasoning models who produce long reasoning traces. By adapting the Recursive Feature Machine (RFM) algorithm -- which can be computed efficiently -- with a probe-informed initialization, we are able to identify the multi-dimensional refusal subspace in seconds, on reasoning (Qwen 3) and non-reasoning (Qwen 2.5) models. While RFM allows for faster subspace identification, it also showed better performances on the ablation task than its alternatives. More work is planned to better understand the relations between subspaces found by different methods. If confirmed, RFM could be a cheap and scalable complement to existing subspace-extraction methods in LLMs.