EvisStress improves face forgery detection with clearer explanations
EviStress: Evidence Stress Learning for Generalizable Face Forgery Detection with Textual Explanations
Computer Vision and Pattern Recognition
Summary
Fake face images are spreading quickly online, making it hard to know what is real or fake. The authors found that typical detectors rely too much on only a few small parts of an image, which makes them less reliable with new types of fakes or changes in images. They created a method called EviStress that stresses the parts the detector depends on and teaches it to still recognize fakes robustly. Their system can also explain why it thinks an image is fake in simple text, helping users understand the decision.
What this means in practice
- •For social media platforms: Improve automatic detection of face forgeries across diverse and evolving fake generation methods with clearer textual explanations for human reviewers.$Commercial implications: Enables development of forgery detection tools that provide trustworthy alerts and explanations, improving content moderation services.
- •For digital forensics teams: Assist forensic analysts by providing robust face forgery identification that remains effective on unseen manipulations and offers easy-to-understand explanations.
Authors
Jiuyao Jing, Xuanyu Yi, Yu Zheng, Chunlei Peng
Abstract
The rapid spread of face forgeries on the Web challenges the credibility of visual content. In practical Web environments, manipulated media originate from evolving generation pipelines and undergo diverse processing during dissemination, making reliable authenticity assessment under unseen manipulations and visual changes a critical challenge. We observe that a standard LoRA-adapted CLIP detector exhibits uneven patch-level decision dependence, concen trating a disproportionate share of class-specific relevance on a small fraction of visual patches. To improve robustness to variations in such highly dependent evidence, we propose EviStress, a two-stage framework for generalizable face forgery detection with textual explanations. In the first stage, Evidence-Targeted Patch Perturbation (ETPP) estimates patch dependence from global, relational, and semantic cues and assigns positively correlated perturbation probabilities, explicitly stress-testing highly dependent visual evidence during training. We further introduce Discriminative Variation Preservation (DVP), which constrains the residual between clean and perturbed representations to retain authenticity discriminative information. In the second stage, Intrinsic Evidence Instruction Tuning (IEIT) transfers the adapted CLIP encoder into LLaVA as its vision tower and performs parameter-efficient multimodal instruction tuning, enabling authenticity prediction and textual explanation directly from the learned forgery-sensitive representation without an external detector. Experiments demonstrate improved cross-dataset and cross-method generalization and corruption robustness, with the multimodal model providing authenticity predictions and human-readable explanations.