AI generated videos are now as hard to spot as real ones
DF26: We Cannot Tell Fake From Real Anymore
Computer Vision and Pattern Recognition
Summary
It is becoming almost impossible for people to tell whether a video is real or created by artificial intelligence. The authors introduced a new test called DF26, which includes a large number of videos showing people speaking—some real and many made by AI models. They found that even advanced computer programs designed to detect fake videos often fail, performing no better than guessing. This shows that current ways of testing fake video detection are not good enough and need to be improved to keep up with new AI technology.
AI-generated videosdeepfake detectiontext-to-video modelsimage-to-video modelssynthetic mediabenchmarkdistribution shiftvideo forgerypublic-speaking videosrobustness
Authors
Severyn Shykula, Andrii Yermakov, Ivan Samarskyi, Dmytro Mishkin, Jan Cech, Anastasiia Mishchuk
Abstract
We introduce DF26, a novel benchmark for detecting AI-generated videos containing fully synthetic clips produced by recent text-to-video and image-to-video models. The videos capture single-person public-speaking scenarios, spanning direct-to-camera recordings, official statements, and studio interviews - 271 real and 2,420 synthetic videos generated by seven modern video models. The study on DF26 shows that human performance in detecting AI-generated videos, as well as state-of-the-art deepfake detectors, is close to random chance. Our results highlight the limitations of current evaluation protocols and motivate the need for benchmarks that explicitly measure robustness to modern generative model distribution shifts.