LLMs adapt internal controls to reflect multiple human values at once
Adaptive Multi-Value Control in LLMs via Causal Activation Steering
Machine Learning
Summary
Large language models need to follow many social and moral rules at the same time, but previous methods only focused on one rule at a time or used fixed settings. The authors introduce AIMES, a way for these models to adjust how strongly they follow different rules while they work, by watching their own internal state. This method adapts continuously during response generation, leading to better performance and smaller changes inside the model. Their tests show that this adaptive approach works better than fixed control methods across different models and value combinations.
What this means in practice
- •For ai product developers: Control multiple social and moral values simultaneously in language model outputs with adaptive internal adjustments during inference.$Commercial implications: Enables developers to sell safer and more aligned AI chat or assistant products that better handle complex human values in real time.
- •For content moderation teams: Deploy adaptive steering to ensure language models consider combined norms and values dynamically, improving response appropriateness in moderation tools.
Authors
Payel Bhattacharjee, Ravi Tandon
Abstract
Large language models (LLMs) are increasingly deployed in settings where responses must reflect multiple, potentially interacting social norms and human values. Activation steering offers a lightweight alternative to training-based alignment by modifying internal activations at inference time. However, prior human-value steering methods have largely considered values in isolation, while direct composition of multiple directions relies on fixed intervention strengths that cannot respond to the model's evolving internal state. Motivated by this key observation, we introduce AIMES, a framework for adaptive multi-value activation steering. AIMES constructs layer-specific bipolar directions for moral-foundation values and uses intermediate-layer vocabulary readouts as online observers. An observer-guided controller then adapts the strength of each requested value intervention at every decoding step based on its current observed state, without training a separate value-state estimator. Across multiple instruction-tuned model families, value combinations, and intervention depths, we find that multi-value controllability varies across both value combinations and intervention locations. Compared with fixed joint steering and prompt-based steering, AIMES shows depth-dependent advantages that are broadly supported across two independent evaluators, with some variation in the precise depth at which specific control effects emerge. These advantages come with smaller realized activation-space interventions than fixed-joint steering and comparable response quality. Overall, our results suggest that online observer feedback can provide lightweight, state-aware adaptation for single-pass multi-value steering.