Safety checkers for AI answers can be tricked by style changes
Style Over Substance: Content-Invariant Wrappers Flip LLM Safety-Judge Verdicts
Artificial Intelligence
Summary
AI safety systems are designed to decide if a response from a language model is harmful. This paper finds that some of these systems do not focus solely on the content but can be confused by how the response is presented, such as adding a polite or educational tone. The authors tested this by keeping the harmful content exactly the same but changing only the style around it, and they found some systems wrongly marked harmful replies as safe. This shows that some AI safety checks can be fooled and that their judgments are not always reliable. The researchers provide data and tools for others to study this issue further.
language modelAI safetyjailbreakcontent-invariant wrapperverdict flipLlama GuardGPT-4omodel evaluationprompt engineeringhuman validation
Authors
Yongxi Zhou, Wenbo Ye, Yuanzhe Liu, Zihan Dong, Junwei Yao
Abstract
Automatic safety judges -- systems such as Llama Guard or a GPT-4o grading prompt that decide whether a model's reply is harmful -- produce the numbers behind almost every reported jailbreak success rate, defense evaluation, and safety leaderboard. We ask whether these judges grade what a reply contains or how it sounds. We keep a reply's content fixed and add content-invariant style wrappers: fixed strings placed before or after the reply that change only its tone (an educational disclaimer, a fake safety "reasoning" block, a token refusal followed by the unchanged harmful body), or, on harmless refusals, framing that merely sounds dangerous. The body is preserved byte-for-byte, so a faithful judge must return the same verdict, and any flip is an error of the judge, not a change in safety. Over 600 JailbreakBench replies x up to 7 forms x 8 judges, we measure flip rates with paired significance tests and measured noise floors. Findings are precise rather than universal: most judges barely move, but specific judges harbor cheaply exploitable blind spots. A token-refusal wrapper flips 19.9% of GPT-4o-mini's correct "unsafe" verdicts (noise floor 0.5%; 18.2% under majority-of-three re-scoring) yet moves Claude only 0.4%. The deployed Llama Guard 4 is deterministically gamed: an "educational course" framing flips 12.3% of its harmful verdicts to safe. A second deployed guard (gpt-oss-safeguard-20b) is immune, and rewriting only the grading prompt (StrongREJECT-style) cuts the attack tenfold on the identical model -- the vulnerability lives in the judge, not the content. A two-annotator human validation confirms 100% content invariance and 90% of flips as judge errors (kappa 0.95-1.0), and a bootstrap shows the underlying model ranking is already unstable to sampling alone. We release the dataset, wrappers, code, and per-verdict labels.