Why Are LLM Backdoor Defenses Fragmented? A Feature-Level Explanation with Sparse Autoencoders
2026-08-31 • Cryptography and Security
Cryptography and Security
AI summaryⓘ
The authors study hidden triggers called backdoors in large language models, which can make the models behave badly when presented with specific inputs. They use a tool called sparse autoencoders to analyze how these backdoors affect the model's internal workings, identifying four types of features involved. Their findings show that different backdoor types (dirty-label vs. clean-label) use different feature patterns, which explains why current defenses don't work equally well for all attacks. They also test a defense method that reduces the success of these attacks without hurting normal model performance.
Backdoor attacksLarge language modelsDirty-label attackClean-label attackSparse autoencodersFeature-level analysisLogit shiftsModel defensesInference-time feature clampingAttack success rate (ASR)
Authors
Yizhe Zeng, Chenxu Niu, Wei Zhang, Hao Huang, Yunpeng Li, Dongxu Han, Dan Du, Cheng Hong, Hequn Xian, Yuling Liu
Abstract
Backdoor attacks pose a serious threat to large language models (LLMs), but existing defenses remain fragmented, failing to pro?vide unified defense against both dirty-label and clean-label attacks. To investigate why such fragmentation arises, we present the first systematic feature-level mechanistic analysis of LLM backdoors using sparse autoencoders (SAEs). Starting from a 2 x 2 comparison of clean and poisoned models on clean and triggered inputs, we trace backdoor-induced logit shifts to high-contributing SAE features and categorize them into four roles: interac?tion, suppressed, mixed, and weight-modified features. This taxonomy reveals system?atic encoding differences: dirty-label back?doors are dominated by isolated interaction features, whereas clean-label backdoors rely more on heterogeneous mixtures of mixed and weight-modified features. These differ?ences explain why existing defenses remain fragmented across attack paradigms. We val?idate this hypothesis through inference-time feature clamping, which reduces ASR to at most 10.8% in most dirty-label settings and at most 15.4% in the majority of clean-label settings, while preserving benign-task perfor?mance. These results show that SAE-based analysis can explain defense fragmentation and guide interpretable backdoor mitigation.