Pre-optimization difficulty predicts effort for person-vanishing attacks
Can Attack Difficulty Be Characterized Before Optimization? A Study of Pre-optimization Difficulty in Person-Vanishing Attacks
Computer Vision and Pattern Recognition
Summary
Adversarial attacks try to fool object detectors, but some images are harder to attack than others. This paper asks if we can guess how hard it will be to attack an image before actually trying. The authors create a fast way to estimate this difficulty based on the image's geometry. Their method helps save computing effort by focusing more on harder images, improving success rates and efficiency.
What this means in practice
- •For security engineers: Optimize adversarial testing by estimating attack difficulty to improve efficiency and success rates when testing person detectors.
- •For machine learning developers: Use pre-optimization difficulty estimates to allocate computational resources efficiently when creating adversarial attacks against detection models.
Authors
Jingyao Xu, Dongdong Wang, Siyang Lu
Abstract
Adversarial attacks against object detectors are traditionally studied from an optimization perspective, where attack difficulty is regarded as an outcome observed only after adversarial optimization. This raises a fundamental question: \emph{can the relative attack difficulty of different inputs be characterized before optimization begins?} In this paper, we investigate this question for person-vanishing attacks by introducing the concept of pre-optimization attack difficulty, which captures intrinsic differences in optimization effort across input images. To estimate this latent difficulty before optimization, we propose Quad-CLEVER, an efficient geometry-based estimator derived from a quadratic approximation of the local person-vanishing margin along the most attack-relevant direction. Extensive experiments across multiple attack algorithms demonstrate that Quad-CLEVER consistently correlates with the observed optimization cost, providing empirical evidence that attack difficulty exhibits a predictable pre-optimization structure. Building upon this finding, we further propose a difficulty-aware attack framework that leverages the estimated difficulty to adaptively allocate optimization budgets for a base attack under a fixed computational budget. On BDD100K, the proposed framework improves the image-level attack success rate by up to 5.78$\%$ while reducing the average optimization cost by up to 11.42 iterations. On the more challenging EventPed dataset, it saves 2.25 optimization iterations while maintaining comparable attack performance. These results demonstrate that attack difficulty can be meaningfully estimated before optimization and that exploiting such estimates enables more computationally efficient adversarial attacks.