Cancerous backdoor gradually corrupts large language model reasoning
Cancerous Backdoor Attack: Gradual Activation via Invisible Tokens and A Stage Controller
Cryptography and SecurityArtificial IntelligenceComputation and Language
Summary
Backdoor attacks on language models usually cause a sudden change in behavior when a hidden trigger is present. The authors show a new kind of backdoor that slowly changes how the model reasons over several steps, making it harder to detect. They do this by hiding invisible tokens that control different attack stages without showing up in user input. Their tests show this gradual corruption happens in different models and tasks better than previous attacks. This means just checking if answers are correct is not enough to find these attacks.
What this means in practice
- •For machine learning security teams: Detect and mitigate multi-stage backdoor attacks that gradually corrupt language model reasoning during inference.
- •For ai system auditors: Improve auditing techniques by accounting for backdoors that activate over multiple interaction stages rather than single triggers.
Authors
Tahmid Imtiaz Imu, Emama Nahid, Prodipta Sen Amartya, Ahsanul Jobayer, Yutong Song, Jiang Wu, Pengfei Zhang, Honghui Xu
Abstract
Backdoor Attacks on Large Language Models (LLMs) typically follow a trigger-response pattern, where satisfying a trigger condition activates a single terminal behavior. However, this single-point backdoor activation does not explore attacks in which malicious behavior develops gradually through multiple intermediate stages before reaching the terminal activation stage. We present Cancerous Backdoor Attack, that gradually corrupts an LLM's reasoning over a sequence of interactions by leveraging invisible tokens and a stage controller. Specifically, invisible tokens provide hidden stages to the model without appearing in the user's visible request and the stage controller tracks the attack's progression during inference to realize gradual activation. We evaluate Cancerous Backdoor Attack across several reasoning datasets. The attack consistently produces gradual reasoning corruption across models and tasks. We compare Cancerous Backdoor Attack with prior backdoor attack baselines to demonstrate our proposed attack outperforms the baselines. Our results indicate that reasoning corruption can develop gradually before it becomes visible, making answer accuracy alone insufficient for detecting backdoor attacks.