Adaptive method finds diverse attacks on black box AI models
ATLAS-AL: Adaptive Trust-Region for Latent Adversarial Searches via Active Learning
Machine LearningCryptography and SecurityComputer Vision and Pattern Recognition
Summary
Testing AI systems by checking if they fail under fixed tests is not enough to ensure their security. The authors introduce ATLAS, a method that actively searches for groups of inputs that cause AI models to make mistakes, even when the model’s inner workings are unknown. ATLAS uses smart sampling and modeling techniques to find and explore these tricky inputs efficiently. It performs better than previous methods on standard image recognition tests, helping developers understand and improve AI safety.
What this means in practice
- •For machine learning engineers: Identify diverse adversarial examples efficiently to evaluate AI model robustness during development.
- •For security audit teams: Continuously monitor AI systems for emerging vulnerabilities with an adaptive black-box adversarial testing tool.
Authors
Marsalis Gibson, Claire Tomlin, Shankar Sastry
Abstract
Security evaluation of learning-based systems requires more than just testing the system against a fixed collection of attacks. It requires adaptive mechanisms that can efficiently discover \textit{sets} of inputs that induce model failure. We introduce ATLAS (Adaptive Trust-Regions for Latent Adversarial Searches), which is a query-based framework that discovers adversarial input sets for black-box learning systems. ATLAS casts attack generation as an active learning level set estimation problem then combines calibrated approximations with a local-global sampling architecture to find regions of the input space that contain adversarial examples. Once discovered, ATLAS is designed to sample points within these adversarial regions to build adversarial sets that accurately represent the state of robustness of the target model. When applied on toy experiments, we find that ATLAS is able to recover more of the adversarial region under a limited query budget than does previous work. When applied to standard and adversarially trained MNIST, CIFAR, and ImageNet model targets, ATLAS produces better representative attacks than other query-based black-box attacks (NES, SignHunter, BayesOpt). ATLAS represents an automated red-teaming framework that can be used for both analyzing the robustness of learning-based systems under development and continuous auditing to see how the robustness of a system changes over time.