Feature learning in neural networks increases risk of backdoor attacks
Why Backdooring Neural Networks is so Easy?
Machine LearningCryptography and Security
Summary
Backdoor attacks sneak hidden triggers into AI models without affecting their normal work, which is a security risk. The authors show that how neural networks learn features can make these attacks easier, even when very few malicious samples are added. They provide a mathematical explanation for why this happens, showing that more advanced learning methods need weaker triggers to succeed. This means current security checks that assume simpler models might underestimate these risks.
What this means in practice
- •For ai security teams: Assess the vulnerability of feature-learning AI models to backdoor attacks beyond linear approximation methods.
- •For machine learning engineers: Design training processes that consider reduced trigger strength requirements when adding backdoor resistance in feature-learning networks.
A theory result. No direct application yet.
Authors
Issam Seddik, Mohamed El Amine Seddik
Abstract
Securing modern AI systems against backdoor attacks remains an open challenge and requires fundamentally principled estimates of the adversary's budget -- the poison fraction $π$ and trigger strength $α$ needed to construct successful yet stealthy attacks. Motivated by recent empirical evidence that poisoning large language models can require a nearly constant number of malicious samples even as clean datasets grow, we derive an exact closed-form analysis of a quadratic neuron trained on a poisoned Gaussian mixture. We show, perhaps counterintuitively, that the same feature-learning dynamics that make neural networks powerful can also make them more vulnerable to backdoors. Specifically, with clean accuracy preserved to first order, $O(π)$, we demonstrate that lazy learning imposes the inverse-square-root scaling $α\propto π^{-1/2}$ for a successful attack, while feature learning induces a quadratic detector whose loss margin scales as $O(α^4)$, improving the attack budget to $α\propto π^{-1/4}$. Consequently, nonlinear feature learning substantially reduces the trigger strength required at small poison fractions, thereby in a sense making feature learners more backdoor vulnerable. These results provide a theoretical mechanism consistent with large-scale empirical observations and demonstrate that security audits based on linear heuristics can systematically underestimate backdoor vulnerability in the widely adopted feature-learning regimes.