Region level deepfake detection struggles with partial face manipulation
RLDF: A Region-Level Deepfake Dataset and Benchmark for Partial-Face Manipulation Detection
Computer Vision and Pattern Recognition
Summary
Deepfake detectors usually look for fake faces that are entirely swapped or generated, but they aren’t good at spotting fakes that only change a small part of a face, like an eye or mouth. The authors created a large new dataset, called RLDF, with millions of images where only parts of the face are swapped, to test and improve these detectors. They found that most current models perform worse when detecting these partial face fakes. They also introduced a new method that uses guidance about which face part was changed to better detect these partial fakes.
What this means in practice
- •For security teams: Improve identification of partially manipulated faces in digital media to enhance deepfake verification processes.
- •For video forensics analysts: Use region-level deepfake detection tools to locate specific altered face regions, aiding more precise forensic investigations.
Authors
Shukesh Reddy, Abhijit Das
Abstract
Existing deepfake detection models are built upon manipu- lated images that are tampered with either exclusively fully swapped or fully synthesised faces, which leads us to an open question: Can off-the-shelf deepfake models detect manipulation effectively while tampering is confined to a local facial sub-region? To fill this gap, we introduce RLDF (Region-Level Deepfake), the first dataset for region-level deepfake evaluation. Employing the proposed region-aware partial swap synthesis model, we generate over 2M par- tial face-swap images by using eight face-swapping meth- ods. Altogether, we considered seven face regions, among which there are four discrete regions (eye, nose, mouth and eyebrow) and four composite facial regions (eye and nose; eye, nose and mouth; eye, nose, mouth and eye- brow; and full face). In addition, we define three evalu- ation protocols and benchmark 12 state-of-the-art detectors across the five popular standard cross-dataset evaluations, DF40, and the proposed RLDF, providing a comprehen- sive audit of where and why existing detectors underper- formed. Our evaluation reveals that existing detectors fail to generalize consistently across partial facial regions, re- sulting in an overall drop of 10.5% AUC and 10% accu- racy under region-level evaluation. Further, we propose a mask-guided cross-attention model as a baseline approach that can effectively detect region-level manipulation. The proposed dataset, code, and checkpoints are available at https://github.com/RLDF-Dataset/RLDF.