Eye tracking helps judge realism of AI videos by showing viewers gaze patterns
Evaluating Visual Realism of AI-Generated Videos with Eye Tracking
Human-Computer InteractionArtificial IntelligenceComputer Vision and Pattern Recognition
Summary
People usually say whether AI-made videos look real by giving ratings or saying yes or no, but that doesn't show how they look at the videos to decide. This paper studied eye tracking, which follows where viewers look on the screen, as another helpful way to understand what makes a video seem real or fake. They tested 90 people watching over 400 AI videos and found eye movements matched how people rated the videos. They also made a tool that can guess how real a video looks using eye tracking data with about 71% accuracy.
What this means in practice
- •For video platform developers: Integrate eye tracking signals to better detect if AI-generated content appears realistic to viewers.
- •For advertising teams: Use viewer gaze data to fine-tune AI video ads for more authentic appearance and engagement.
Authors
Danqing Shi, Lan Jiang, Jake Lyons, Miri Zilka
Abstract
Human evaluation of AI-generated videos has so far relied on explicit human feedback, such as judgments or ratings. While these signals capture the conclusions viewers reach, they fail to offer insights into the perception that leads to them. This paper investigates eye tracking as a complementary human evaluation signal for AI-generated videos. We conducted two Visual Turing Test (VTT) experiments with a total of 90 participants on videos generated by latest generative models, in which participants watched each video twice and then gave their feedback. Across the experiments, we collected 4,286 valid eye-tracking trials on 438 videos. Our analysis indicates that gaze patterns offer informative evaluation signals that align with explicit human feedback. Building on the findings, we propose the first computational method GazeEval to predict the perceived visual realism of AI-generated videos from eye-tracking data. It achieves 71.3% accuracy in prediction, providing evidence that eye tracking data carries informative signals about how people evaluate a video. We release the study data to support further research on eye tracking for evaluating AI-generated video.