Attack success depends on similarity between deepfake detectors
What Makes Adversarial Examples Transfer Across Deepfake Detectors?
Computer Vision and Pattern RecognitionArtificial IntelligenceCryptography and SecurityMachine Learning
Summary
Deepfake detectors try to spot fake images or videos, but they can be tricked by specially altered inputs called adversarial examples. This paper looks at why these trick inputs sometimes work across different detectors and finds that the more alike two detectors are in design or training, the more likely the trick works from one to the other. They tested many detectors and attacks to see how factors like model type and training data affect this transfer. Their results show combining attacks from multiple sources can reveal significant vulnerabilities that single-source tests might miss.
What this means in practice
- •For security engineers: Evaluate deepfake detector robustness by testing attacks crafted on models sharing architecture or training data with the target to better predict true vulnerability.
- •For adversarial machine learning teams: Develop multi-source adversarial attacks to improve detection of weaknesses in deepfake detectors beyond single-model evaluations.
Authors
Rafael M. Mamede, Pedro C. Neto, Ana F. Sequeira
Abstract
Deepfake detectors remain vulnerable to transfer-based black-box attacks, in which adversarial examples are generated on a source surrogate model and transferred to a target model, unknown to the attacker. Yet how source--target compatibility shapes attack success remains poorly understood. Prior studies evaluate limited detector pools and rarely disentangle architectural from training factors. We conduct a controlled evaluation of adversarial transferability across 60 detectors spanning six backbones, two pretraining regimes, and five training-data configurations, using two attack procedures: AutoAttack (AA) and the Carlini--Wagner attack with Expectation over Transformation (CW--EOT). Matched comparisons reveal significantly higher transfer when source and target share an exact backbone, architecture family, pretraining regime, or training data. This compatibility structure is attack-dependent: exact backbone compatibility has the largest effect under AA, whereas shared pretraining and training data have the largest effects under CW--EOT. When transfer is averaged across non-target sources, mean attack success rate (ASR) is $7.21\%$ under AA and $19.52\%$ under CW--EOT. By contrast, a multi-source oracle combining both attacks attains a \(64.48\%\) mean ASR after excluding exact backbone and training-data matches, showing that source averaging can substantially understate target vulnerability. We release 240,000 adversarially perturbed images, complete pairwise transfer results, detector configurations, and evaluation code. These findings establish source--target compatibility and source-model selection as central dimensions of credible transfer-based black-box robustness evaluation.