Deepfake detection improves by using detailed face shape data
MOTIF: Person-of-Interest Deepfake Detection Beyond 3DMM Coefficients
Computer Vision and Pattern Recognition
Summary
Deepfake videos that mimic a specific person are a big problem. Many detectors try to spot fakes by looking at a simplified 3D face model, but it wasn’t clear which parts work best. The authors found that just the shape info in the model is enough to catch most fakes, and using the full detailed face surface adds even more accuracy. They combined these insights into a new detector called MOTIF, which works better than previous methods without needing fake videos or person-specific training.
What this means in practice
- •For video content moderators: Detect videos faking specific individuals more accurately without relying on manipulated training data.
- •For security monitoring teams: Improve investigation tools by incorporating detailed 3D face surface data to better identify deepfakes targeting public figures.
Authors
Giovanni Affatato, Sara Mandelli, Paolo Bestagini, Stefano Tubaro
Abstract
Video deepfakes targeting a specific individual, the Person-of-Interest (POI), are the most harmful ones, and, since a public figure is abundantly recorded, a detector can be built from genuine footage of that individual. Such detectors commonly describe a subject through a 3D Morphable Model (3DMM) and adopt its coefficients as a whole, so which part of that description carries the signal has never been measured. We dissect it, holding the encoder, the training corpus and the enrollment protocol fixed and varying only what the encoder observes. The groups of coefficients prove largely redundant, since the shape block alone recovers almost all the accuracy of the full vector, and their temporal evolution contributes a real but bounded amount. We further show that the dense surface the same fit returns, which these detectors discard, carries identity information that the coefficients do not, and that it helps precisely where they are weakest. We assemble the best configuration into MOTIF, a visual-only detector trained on real videos only, with no manipulated video and no POI-specific data. It improves on both state-of-the-art POI detectors in every dataset and manipulation of our benchmark and at two quality levels. Our experimental code will be released at https://github.com/polimi-ispl/MOTIF.