Large language models get safer at handling multiple harm types
How to Tame a Multi-Headed Hydra? Adaptive Multi-Category Safety Steering for Large Language Models
Cryptography and SecurityComputation and LanguageMachine Learning
Summary
Large language models sometimes respond in unsafe ways when given harmful prompts involving different types of bad content. The authors created a new method called CAM-Steer that helps the models detect multiple kinds of risks at once and adjust their responses to be safer. CAM-Steer looks at the model’s internal state and rotates it in a way that reduces harmful content without changing the model itself. Tests showed CAM-Steer works better than previous methods, especially when different harm types happen together.
What this means in practice
- •For llm safety engineers: Improve real-time detection and mitigation of multiple safety risks in large language models without retraining or parameter changes.
- •For chatbot developers: Reduce unsafe or harmful chatbot responses when prompts involve several categories of risky content simultaneously.
Authors
Chenxi Wang, Ruiyang Huang, Li Huang, Yifan Wu
Abstract
As large language models (LLMs) become increasingly widespread, preventing unsafe responses to harmful prompts is essential for their safe deployment. Activation steering offers an approach to improving LLM safety by modifying internal activations during inference without updating model parameters. However, a single prompt can involve multiple harm categories, and steering toward safety in one category may leave harmful content from another unaddressed. Despite advances in adaptive steering, existing methods do not explicitly coordinate steering direction and strength when multiple harm categories co-occur within a single prompt. To address this problem, we propose CAM-Steer, a Category-Adaptive Multi-category Safety Steering framework. Specifically, it estimates the risk associated with each harm category by comparing the current hidden state with safe and unsafe prototypes. The estimated risks are then used to combine the safety directions for different harm categories into a single steering direction and to determine the strength of the intervention. Finally, it rotates the hidden state along the composed steering direction, with the rotation angle determined by the estimated risks, while preserving the hidden-state norm. Experiments across three LLM backbones and seven harm categories show that CAM-Steer outperforms the evaluated baselines in average defense success rate, including when categories co-occur. Further analyses support its component designs and informative risk scores, with negligible inference overhead.