Activation flow method improves controlled model behavior without fine-tuning
Activation Flow: Manufacturing Activations for Steering
Machine LearningArtificial IntelligenceComputation and Language
Summary
Some AI models can hide their true abilities by underperforming on purpose, making it hard to steer them toward correct answers. The researchers developed Activation Flow (ActFlow), a way to create the activations needed to guide these models using only a few correctly labeled examples, without changing the model itself. ActFlow adjusts certain internal signals so that the model ranks the right answers highest. Tests showed this method significantly improved performance on locked models, nearly matching the results from fully fine-tuned ones, and worked better than other approaches to steering.
What this means in practice
- •For machine learning engineers: Enable steering of locked or sandbagging models using limited labeled data without costly fine-tuning.
- •For ai system integrators: Unlock models secured by prompt or parameter locks to improve task accuracy without modifying model weights.
Authors
Hong Kiat Tan, Linh Le, David Williams-King
Abstract
Difference-in-means steering requires activations recorded while a model shows the desired behavior, which a sandbagging model withholds by deliberately underperforming. We introduce Activation Flow (ActFlow), which manufactures these activations from $k$ correct labels without fine-tuning. ActFlow sets target logits that rank each labeled item's correct answer first, and moves the logits toward them by adding one vector $x$ to all $k$ residual streams at one layer. ActFlow is a family of ordinary differential equations for $x$, one for each rule that maps the required logit change to the velocity of $x$. The smallest-norm rule lands exactly on the targets, while the others keep only the top singular directions of the Jacobian. We test ActFlow on three instruction-tuned models, each locked by a sandbagging prompt and by a password-locked LoRA. At $k=40$, ActFlow keeping five singular directions raises the mean held-out ARC-Easy accuracy over the six locked models from $0.05$ to $0.85$, against $0.88$ for fine-tuning and $0.92$ for the honest models. Furthermore, it scores higher than the smallest-norm rule in 16 of the 18 combinations of locked model and $k$, and its steering direction is nearly orthogonal to the honest difference-in-means direction. It also unlocks two LoRA locks where the honest direction fails.