Language model jailbreak attacks improve by targeting harmful response content
ECHO: Replacing the Compliance Proxy with Response Harmfulness in Optimization-Based Jailbreak Attack
Cryptography and SecurityComputation and Language
Summary
Large language models are sometimes tricked into giving harmful answers despite safety checks. Previous attack methods tried to guess harmfulness by looking at the prompt but not what the model actually said. The authors developed ECHO, a method that directly scores how harmful the model's full response is, improving the ability to find harmful outputs. They tested ECHO on several open-source models and found it more effective at generating harmful replies than earlier methods.
What this means in practice
- •For ai safety engineers: Evaluate and improve language model defenses by using ECHO to detect harmful responses more accurately in a black-box setting.
- •For security analysts: Test deployed language models for vulnerabilities by simulating jailbreak attacks focused on harmful outputs with ECHO.
Authors
Yuanwei Wu, Lingdong Huang, Yunlong Mao, Kanye Ye Wang, Jingyu Hua, Sheng Zhong
Abstract
Despite extensive safety alignment, large language models remain vulnerable to jailbreak attacks. Existing optimization-based attacks share a single objective: maximize a token-level proxy of victim compliance---such as the likelihood of an affirmative prefix---computed on the prompt rather than the decoded reply. This proxy cannot distinguish a cautious hedge from a fully actionable harmful answer, leaving the search blind to what the model actually says. We propose ECHO (Evaluation-Conditioned Harmfulness Optimization), which replaces the compliance proxy with response-level, continuous harmfulness scores from HarmSig, a 1.7B regressor grounded in actual harmful capability. ECHO couples HarmSig with five word-level perturbation operators and Keyword-Anchored Beam Search with global top-m memory, under a strict black-box threat model. Across five open-weight victims on AdvBench, ECHO achieves up to 0.898 StrongREJECT mean score and 89.23% HarmBench success. Our code is publicly available at https://github.com/STANWYW/echo.