Investigating and Alleviating Harm Amplification in LLM Interactions
2026-06-01 • Computation and Language
Computation and LanguageMachine Learning
AI summaryⓘ
The authors study how large language models (LLMs) can unintentionally help people create harmful content through ongoing conversations, making dangerous actions easier and faster. They created HarmAmp, a test set showing twelve types of risky scenarios where harm builds up over multiple dialogue turns. To tackle this, the authors developed TrajSafe, a system that watches conversations, guesses when harm might happen, and tries to steer the chat to safer outcomes. Their tests show TrajSafe can reduce harmful responses without blocking too much or hurting the model's normal abilities. Overall, the authors offer a new way to make LLMs safer during complex interactions.
Large Language ModelsHarm AmplificationMulti-turn ConversationsSafety MonitoringHarmful ContentBenchmarkTrajectory PredictionUser Intent ProbingContent ModerationOperational Specificity
Authors
Ruohao Guo, Wei Xu, Alan Ritter
Abstract
Large language models (LLMs) can serve as helpful assistants, yet they can equally function as harm amplifiers that enable malicious users to achieve harmful outcomes beyond their capabilities through extended interactions. This risk manifests along two axes, i.e., democratizing domain expertise that allows novices to produce specialized harmful content, and scaling harmful operations at volumes that manual effort cannot match. Existing works, however, often overlook how LLMs compound harm across multi-turn conversations. We introduce HarmAmp, a new benchmark for multi-turn harm amplification scenarios spanning twelve risk categories. Each scenario is grounded in real-world threats and satisfies rigorous criteria, i.e., substantive amplification, operational specificity, and multi-turn necessity. We further propose TrajSafe, a proactive monitor that anticipates harmful trajectories and intervenes through actions such as probing users' genuine intents and steering the models towards safer completion. Our extensive experiments demonstrate that TrajSafe significantly reduces the harmfulness incurred in multi-turn interactions while preserving a low over-refusal rate and the target model's general capabilities. Our work offers a promising paradigm to alleviate the nuanced safety risks in LLM interactions.