Deepfake detection improves by analyzing image reconstruction patterns
Spectra: Multi-Scale Implicit Decomposition for Deepfake Detection
Computer Vision and Pattern RecognitionArtificial Intelligence
Summary
Deepfake videos and images are becoming more common and harder to detect using current methods, which often focus on the image itself or its frequency patterns. The authors present a new approach called Spectra that looks at how an image is reconstructed by a neural network, revealing hidden clues that can help distinguish fake from real images. This method combines detailed reconstruction patterns with visual features to better detect forgeries, even when fakes are generated by new techniques or compressed. Their approach also works well for detecting other types of facial attacks, suggesting it could be a general tool for verifying face authenticity.
What this means in practice
- •For security software developers: Improve fake media detection tools by incorporating reconstruction-based signals to detect deepfakes across diverse generation methods and compression settings.
- •For biometric system designers: Enhance face authentication systems with a unified detection method that identifies deepfakes and other face attacks using reconstruction artifact analysis.
Authors
Dhananjaya Jayasundara, Kartik Narayan, Vishal M. Patel
Abstract
Deepfakes increasingly undermine trust in digital media, yet existing detectors often struggle to generalize across unseen manipulation techniques, compression levels, and dataset shifts. Most approaches rely on RGB appearance, frequency-domain artifacts, or their combination, learning cues present in the final synthesized image. While effective in-distribution, such cues are frequently generator-specific and can diminish as synthesis models evolve. We argue that a complementary perspective can be obtained through implicit neural representations (INRs), which expose how an image is reconstructed rather than merely how it appears. By fitting an INR to an image and analyzing its internal activations, one can reveal reconstruction dynamics that are difficult to observe in the image domain alone and may capture more intrinsic differences between genuine and synthetic content. To this end, we propose Spectra, a deepfake detection framework that combines semantic visual representations with artifact-aware cues derived from INR reconstruction. Spectra extracts multi-level reconstruction heatmaps from image-specific INRs and jointly models them with semantic features through a bidirectional fusion mechanism. This enables direct interaction between image semantics and reconstruction artifacts, allowing the model to identify forgery cues that are less dependent on specific generation pipelines. Extensive experiments on multiple deepfake benchmarks demonstrate improved robustness and cross-dataset generalization, consistently outperforming detectors based solely on RGB appearance or semantic representations. Furthermore, we show that the proposed representation extends beyond deepfake detection to face presentation attack detection, highlighting its potential as a unified framework for facial attack detection.