Framework certifies machine learning defenses against training data attacks
A Certified Robustness Framework for Backdoor Poisoning Attacks using $f$-Divergences
Machine LearningCryptography and Security
Summary
Machine learning systems can be tricked if someone tampers with the data they learn from. This paper introduces a new method that helps verify when these systems are safe, even if bad data or labels have been added during training. The authors build a general approach that mixes different ways of adding noise and checking for attacks to ensure the model’s predictions stay reliable. Their method can handle many types of attacks and recovers earlier known defenses as special cases.
What this means in practice
- •For machine learning engineers: Guarantee model immunity to specific data poisoning attacks by certifying defense with combined data and noise randomization techniques.
- •For security teams in ai services: Evaluate and certify AI systems’ protection levels against complex training data manipulations using the proposed composable robustness framework.
Authors
Evan Rose, Alina Oprea
Abstract
Machine learning systems trained on untrusted data are vulnerable to data and backdoor poisoning attacks, in which an adversary corrupts training and test samples to manipulate predictions at inference time. Randomized smoothing yields provable defenses against such attacks, but existing applications to train-time robustness are limited to specific threat models and smoothing mechanisms. We propose a general framework that allows expressing complex adversary constraints on the number of corrupted training examples, feature and label perturbation magnitudes, and test-time corruptions. Our framework randomizes inputs by combining data subsampling with feature, label and test-sample noising. We develop a composable certification procedure based on $f$-divergences that handles arbitrary combinations of threat models and noising strategies, and recovers several prior certification regimes exactly as special cases. We demonstrate our framework experimentally by certifying, for the first time, robustness under general $\ell_p$-constrained poisoning adversaries under bagging with additive Laplace, Gaussian, and Uniform noises.