Selecting poison data drastically changes success of llm backdoor attacks
Pick Your Poison: Learning to Select Poison Sets for Stronger LLM Backdoor Attacks
Machine LearningArtificial IntelligenceComputation and LanguageCryptography and Security
Summary
Backdoor attacks sneak harmful instructions into AI language models by adding tricky examples during training. The authors discovered that picking which bad examples to add can make the attack either fail or almost completely succeed, even if the number of bad examples stays the same. They created a method called SAILS that learns to pick the most effective bad examples by testing many sets quickly, making these attacks much stronger. This approach works across different AI tasks and setups.
What this means in practice
- •For security analysts: Evaluate the worst-case impact of backdoor poisoning by selecting optimized poison sets that reveal vulnerabilities missed by random sampling.
- •For ai safety teams: Develop more effective defenses against backdoor attacks by understanding how poison set selection influences model behavior across tasks and deployment types.
Authors
Aashiq Muhamed, Mona T. Diab, Virginia Smith, Andrew Ilyas, Matthew Jagielski
Abstract
Backdoor poisoning attacks add poisoned examples to otherwise-clean finetuning data, pairing a trigger with a target behavior that the model learns to produce when the trigger appears. Existing evaluations typically fix the number of poisoned examples and sample them at random from a candidate pool. We show that this can severely underestimate worst-case vulnerability: across three LLaMA-3-8B backdoor settings, holding the model, clean data, and poison count fixed, attack success ranges from 3% to 80% depending only on which poison set is chosen. We formalize poison selection as oracle-budgeted set optimization and introduce SAILS (Set-level Audit-Informed Iterative Learned Selection), which learns a set scorer from a few hundred finetune-and-evaluate runs, ranks millions of candidate sets, and audits only a small shortlist. SAILS improves held-out attack success by 30 percentage points on average over the strongest influence baselines, transfers from small-scale to full-scale finetuning, and extends to code-generation, agentic, and API-only backdoors.