Instruction based multi face forgery detector improves detection accuracy
IMFD: End-to-end Multi-Face Forgery Detection through Instruction-based Large Vision-Language Models
Computer Vision and Pattern Recognition
Summary
Deepfakes—fake images or videos made by computers—can trick people especially when multiple faces appear at once. Traditional methods check each face alone, missing important clues from the background or nearby faces. The authors created IMFD, a system that teaches a large AI model to look at the whole picture using instructions, finding and checking many faces together. By including info about where faces are in the instructions, IMFD spots fake faces better than earlier tools.
What this means in practice
- •For social media platform security teams: Detect multiple deepfake faces in images posted online by analyzing entire pictures with improved accuracy.
- •For digital forensics analysts: Verify the authenticity of images containing several people by integrating face locations and context in forgery detection workflows.
Authors
Dasom Choi, Sangjun Moon, Hyeongchan Im, Jaeeon Park, Jingun Kwon, Hidetaka Kamigaito, Taro Watanabe, Manabu Okumura
Abstract
The rapid increase of deepfakes has raised significant concerns due to their spread on social media. Traditional multi-face forgery detectors crop and verify each face independently, ignoring background context and inter-face relationships, which often yields suboptimal performance. To overcome these limitations, we leverage instruction-based Large Vision-Language Models (LVLMs), which can interpret entire images and follow complex textual instructions. We propose a simple yet effective single-stage multi-face forgery detector, called IMFD (Instruction-based Multi-face Forgery Detector), which is trained end-to-end to jointly localize faces and predict per-face forgery labels. Rather than treating face box prediction only as a joint objective, IMFD explicitly integrates predicted face bounding boxes into the instruction as visual cues that enhance instruction grounding and forgery detection. To support the training and evaluation of IMFD, we convert existing multi-face forgery datasets into an instruction-based format. Experimental results and analyses show that IMFD improves multi-face forgery detection by integrating face bounding boxes into the instruction, and consistently outperforms various state-of-the-art methods.