Sparse internal signals improve control over harmful AI outputs
SAFETYEDIT: Sparse Internal Targets with Prompt-Wide Control for LLM Jailbreak Backdoors
Computation and Language
Summary
Large language models can be tricked into giving harmful answers when certain triggers are used, creating a safety problem. This paper presents a method called SAFETYEDIT that focuses on changing a very small part of the model’s internal workings to control when harmful responses happen, without affecting normal answers. The authors show that targeting a single internal coordinate combined with a special tracking signal works better than previous methods at blocking harmful outputs when triggered. Additionally, their approach keeps the model’s usual abilities intact on normal inputs.
What this means in practice
- •For ai safety engineers: Use sparse internal model edits to create controlled refusal triggers that prevent harmful AI outputs under specified inputs.
- •For llm developers: Implement compact internal triggers for safer large language models that preserve normal behavior while blocking unsafe responses.
Authors
Yutong Ke, Yansu Wang, Zeyu Zhang, Xihao Guo, Chongwen Zhao, Kaizhu Huang
Abstract
Model-editing backdoors aim to induce harmful responses under a trigger while preserving ordinary behavior on untriggered input. Recent model-editing approaches target affirmative response prefixes through rank-one updates to a small subset of model weights. Yet when this prefix-level target is satisfied, the model may still revert to refusal or produce a coherent, apparently compliant response that does not address the request. To overcome this limitation, existing internal-representation approaches shift focus from output tokens to dense, high-dimensional compliance directions in latent space. Existing representation-level approaches rely on preservation constraints to maintain conditionality, motivating us to ask whether a smaller internal target can offer a different effectiveness-preservation trade-off. Motivated by the comparable control capability of single-coordinate interventions, we investigate a sparse refusal-related MLP coordinate as this target. We propose SAFETYEDIT, which optimizes a single refusal-related coordinate and uses KV-cache marking in early attention layers to propagate a prefix-trigger signal across subsequent instruction positions. On some checkpoints a downstream MLP reader converts this signal into control of the coordinate, while on others the propagated signal alone suffices; the method is defined by the sparse target rather than by a fixed writer-reader pair. Across four open-weight LLMs and three dataset groups, SAFETYEDIT achieves the highest triggered Guard ASR in 11 of 12 comparisons and the highest mean StrongREJECT score in all 12, relative to the baselines evaluated for each model. Crucially, comprehensive ablation and behavioral audits confirm generally minimal changes in untriggered behavior and general capabilities in the tested configurations.