Explainable deepfake detection improves with metric-aligned optimization

Metric-Aligned Group Relative Policy Optimization for Explainable Deepfake Detection

Computer Vision and Pattern Recognition

Summary

Detecting deepfake images is important, but understanding why an image is labeled fake or real is even more useful. The authors created a two-step method: first, a computer decides if the image is real or fake; then, a language model explains the reasons behind that decision in everyday words. They also improved how well the explanations match evaluation criteria by using a special training approach called Group Relative Policy Optimization. Their approach performed best on detailed explanation metrics in a public challenge.

What this means in practice

  • For content moderation teams: Provide clear visual evidence explanations for flagged deepfake images in social media or news platforms to support informed decisions.
  • For digital forensics analysts: Generate concise, non-expert summaries of manipulated images to aid quick understanding and reporting in forensic investigations.

Authors

Luka Dragar, Peter Peer, Vitomir Štruc, Borut Batagelj

Abstract

Deepfake detectors are commonly evaluated on real/fake classification alone, but real-world applications require explanations of the visual evidence behind each prediction. Joint vision-language models struggle with authenticity assessment, producing "real" narratives for manipulated images. To address this, we propose a decoupled predict-then-explain pipeline as our submission to the ACM Multimedia 2026 Explainable Deepfake Detection Challenge, with methods trained and evaluated on the XPlainVerse benchmark. A vision-foundation-model ensemble first classifies the image, then a Qwen3-VL-8B model generates a decision-conditioned explanation of the visual evidence. For fake images, a second Qwen3-VL-8B compresses that text into a short non-expert summary without using the image. We further apply metric-aligned Group Relative Policy Optimization (GRPO) to optimize both explainers against the challenge criteria. Under matched conditions, GRPO raises complex BERTScore-F1 from 0.686 to 0.712 and also improves grounding (entity and evidence F1); our submission ranked first on complex BERTScore-F1 and seventh overall.