GraphVid: Interactive Graph-Controllable Video Generation
2026-07-23 • Computer Vision and Pattern Recognition
Computer Vision and Pattern RecognitionArtificial Intelligence
AI summaryⓘ
The authors address the challenge of controlling video generation with multiple interacting objects by introducing GraphVid, a model that uses graphs to represent object interactions for more precise control. Instead of relying on tricky motion paths, their method uses structured graphs to guide video creation, making it easier to handle complex scenes. They also created GraphVid-Bench, a new dataset with detailed interaction labels to help train their model. Compared to previous methods, GraphVid produces better quality videos with less data and smaller models, showing the value of using structured information in video generation.
video generationmulti-object interactiongraph-conditioned modelimage-to-video generationstructured interaction graphsGraphVid-Bench datasetFID (Fréchet Inception Distance)FVD (Fréchet Video Distance)PSNR (Peak Signal-to-Noise Ratio)SSIM (Structural Similarity Index Measure)
Authors
Vedant Shah, Onkar Susladkar, Tushar Prakash, Kiet Nguyen, Tianjio Yu, Adheesh Juvekar, Muntasir Waheed, Ismini Lourentzou
Abstract
Controllable video generation remains challenging due to the difficulty of specifying precise multi-object interactions using text prompts or motion-control inputs that primarily constrain pixel movement. In practice, trajectory-based control often requires users to draw accurate tracks for multiple objects, which scales poorly with scene complexity and becomes ambiguous under occlusion or overlap. To enable flexible yet precise multi-subject control, we introduce $\textbf{GraphVid}$, a graph-conditioned image-to-video generation model that enables interactive control through structured interaction graphs. We further curate $\textbf{GraphVid-Bench}$, a large-scale interaction-centric video dataset with structured relational annotations to enable training of interaction-aware video generation models. Despite using substantially less training data and fewer trainable parameters than prior motion-control methods, GraphVid delivers strong controllability and video quality. Compared with Motion-I2V, GraphVid reduces FID by up to 39.9% and FVD by 37.6%, while improving PSNR (9.87=>15.98) and SSIM (0.38=>0.61). Our results highlight the potential of structured semantic interfaces as a powerful paradigm for controllable video generation.