Flexible control of large language model traits during use
From Black Box to Controller: Inference-Time Steering of LLM via Feature Amplification
Artificial Intelligence
Summary
Large language models can be hard to adjust when you want them to behave differently, like being more polite or less toxic, without retraining them. The authors introduce NeuroSteer, a tool that can change these behaviors while the model is running by boosting certain internal features linked to traits like politeness or sentiment. This lets users control how strong these behaviors are depending on the situation or user preference. Their tests show it works better than some existing methods but with some trade-offs in text quality depending on the task.
What this means in practice
- •For chatbot developers: Adjust chatbot personality or caution extensively without retraining by amplifying specific model features at inference time.$Commercial implications: Enables customizable commercial chatbot products that adapt behavior dynamically to user or scenario needs.
- •For content moderation teams: Steer large language models to reduce toxicity and bias in automated content generation with adjustable strength controls.
Authors
Keqi Chen, Zekai Sun, Yuhua Wen, Tianyi Tian, Xinyang Huang, Huijun Lian, Yingming Gao, Ya Li
Abstract
Inference-time steering is crucial for deploying Large Language Models (LLMs) in real-world applications, where they must flexibly adjust behavior preferences. For instance, a model should dynamically adjust its caution level based on the risk of the scenario. This requirement introduces a challenge: modulating the intensity of LLM preferences. Prompt engineering can require task-specific prompt design, while fine-tuning requires additional optimization and data. These requirements can make repeated behavioral adjustments costly. To address this, we introduce NeuroSteer, a plug-and-play framework that enables flexible, quantitative adjustment of LLM preferences at inference time. NeuroSteer employs a Sparse Autoencoder (SAE) as its controller. Comparing neuron activation frequencies between positive and negative examples isolates "feature neurons" that correspond to target preferences such as politeness and positive sentiment. During inference, the activations of selected SAE features are amplified in the latent representations to steer the output distribution. Users can adjust the strength of this steering on a per-scenario or per-user basis, granting the LLM fine-grained, quantifiable trait modulation. Our experiments show that NeuroSteer can alter sentiment, toxicity, stance, and politeness at inference time. It achieves higher target-direction steering scores than the compared baselines in several settings, with task- and strength-dependent trade-offs in generation quality.