Steering multiple traits in large language models more clearly and independently
Orthogonal and Parametrized Multi-Attribute Steering of LLMs
Machine LearningArtificial Intelligence
Summary
Large language models can be made to behave differently by tweaking internal signals called activations. The authors found better ways to control these signals more clearly and meaningfully, so changing one trait doesn’t accidentally change others. They introduced methods to set specific behavior strengths and to make different behaviors affect the model independently, allowing multiple traits to be adjusted at the same time without interference.
What this means in practice
- •For ai developers: Adjust multiple traits of language models reliably in deployment without retraining, improving behavior customization.
- •For chatbot designers: Create chatbots that can express distinct multiple personality traits or styles without traits overlapping or diluting each other.
Authors
Remy Ogasawara, Cullen Anderson, Narmeen Fatimah Oozeer, Foad Namjoo, Amir Abdullah, Jens Kristian Refsgaard Schou, Jeff M. Phillips
Abstract
Activation steering offers an effective and efficient way to modify an LLM's behavior at inference time. However, it remains unclear how to apply steering such that the signal is consistently strong and the steering strength has a universally interpretable meaning. We address this first by identifying the most steerable interval for each steering vector and rescaling the steering range accordingly. Building on this, we introduce a more interpretable method for steering, called parameterized steering which instead of pushing activations in a direction, it sets them at the desired trait strength. Next, with multi-trait steering, we find that naively combining vectors causes them to interfere with one another. In response, we develop a method to orthogonalize the vectors; it decouples these traits, allowing a model to exhibit one behavior without inducing the others, thus enabling multiple traits to be steered simultaneously.