Video generation improves subject consistency over long dynamic scenes
Towards Subject Consistency over Dynamic Subject Sets in Video Generation
Computer Vision and Pattern Recognition
Summary
Keeping the same subjects consistent in videos gets harder as videos get longer, especially when subjects appear and disappear at different times. The authors created a new way to evaluate how well video generation keeps track of these changing subjects through their visible time. They tested their method with fake videos that had intentional errors and showed their method finds problems that older methods miss. They also used their evaluation to improve existing video generation models, making generated videos more consistent without hurting other video qualities.
What this means in practice
- •For autonomous vehicle developers: Improve simulated video outputs to consistently preserve vehicle identities throughout extended driving scenes.
- •For animation studios: Generate longer, more consistent animated videos that accurately maintain character appearances as they enter and leave scenes.
Authors
Tongcheng Zhang, Jun Zhu, Jianfei Chen
Abstract
We argue that as video generation extends to longer durations, subject consistency should be evaluated over \textit{dynamic subject sets}. We therefore introduce \textbf{DynSC-Eval}, an evaluation framework that dynamically tracks eligible subjects throughout their visible lifespans and measures local continuity and global identity preservation using six complementary object-level metrics, with explicit detection of inconsistency events. To validate its effectiveness, we design synthetic experiments that actively inject inconsistency events, demonstrating both the sensitivity of DynSC-Eval and the limitations of existing metrics. Evaluations of diverse models on 5s, 15s, and 60s video generation further reveal substantial subject consistency differences that are obscured by conventional metrics. Beyond evaluation, we construct rewards from DynSC-Eval and apply DiffusionNFT post-training in an autonomous-driving testbed. On 5s generation, our approach reduces the six inconsistency metrics by an average of 13.82\% for Wan-2.1-1.3B and 5.66\% for SANA-2B, with improvements also observed on the I2V model ReSim. Qualitative comparisons further demonstrate the effectiveness of our method. We then extend generation to 10s and 30s through curriculum learning and show that consistency optimization remains effective while largely preserving other capabilities.