Opportunistic Target Selection: Early Directional Commitment for Query-Efficient Black-Box Adversarial Attacks
2026-05-25 • Machine Learning
Machine LearningComputer Vision and Pattern Recognition
AI summaryⓘ
The authors found that some black-box adversarial attacks get stuck without focusing on a specific wrong class, making them inefficient. They propose Opportunistic Target Selection (OTS), which helps these attacks switch to targeting a likely wrong class early on without needing extra information. They tested OTS on several attacks and models, showing it improves success rates and efficiency in many cases. However, OTS does not help for attacks that use gradient estimates or models trained to resist attacks, indicating its benefits are limited to certain attack types.
black-box adversarial attacksuntargeted attacktargeted attackfeature spacescore-based attacksImageNet classifiersgradient estimationmargin lossadversarial trainingquery efficiency
Authors
Florent Tariolle, Florian Yger
Abstract
Black-box adversarial attacks that minimize only the ground-truth confidence suffer from class drift: perturbations wander through the feature space without committing to a specific adversarial class, wasting queries on diffuse, undirected progress. We introduce Opportunistic Target Selection (OTS), a lightweight wrapper that switches an untargeted attack to a targeted objective early in its trajectory, locking onto whichever non-true class currently leads. OTS requires no architectural modification to the underlying attack, no gradient access, and no a priori target-class knowledge. We validate OTS on three score-based attacks (SimBA, Square Attack with cross-entropy loss, and Bandits) across five standard ImageNet classifiers (4,500 runs). On random-search attacks, OTS closely tracks oracle performance, with gains up to +27 pp in success rate and 43% relative reduction in censored-mean iterations on ResNet-50. On gradient-estimation attacks (Bandits) and attacks with margin loss, OTS is redundant, a negative result that reinforces our interpretation of OTS as a margin-loss surrogate. On adversarially-trained models, a bimodal difficulty distribution eliminates the regime where targeting helps.