Video generation models improved by balancing temporal state transport
Temporal State Transport in Video Generation: Diagnosing and Correcting Spectral Imbalance
Computer Vision and Pattern RecognitionMachine Learning
Summary
Creating videos with AI requires keeping track of how things change and stay the same over time, like a person's identity or motions. The authors found that existing methods miss when this tracking goes wrong, causing either jumbled or overly mixed frame details. They introduced a new way to measure these problems and a way to fix them without retraining the model. This fix improves the smoothness and quality of AI-generated videos by focusing corrections only on the most problematic parts.
What this means in practice
- •For video ai developers: Enhance pretrained video generation models by correcting temporal inconsistencies without requiring additional training, leading to better video quality.
- •For animation studios: Improve AI-generated animation smoothness by diagnosing and regulating temporal attribute transport to maintain coherent visual details across frames.
Authors
Luyao Tang, Bingjun Luo, Dong Yi, Jialin Guo, Haoning Xi, Cheng Chen, Yizhou Yu, Chaoqi Chen
Abstract
Reliable video generation requires more than high-quality frames to form a coherent story: a model must maintain a persistent state, transporting visual attributes such as identity, scene layout, motion, and fine details across time. Existing training-free methods mainly strengthen cross-frame attention or analyze local attention entropy, but these views do not reveal whether temporal interactions stay in a healthy transport regime. In this work, we study video generation through the perspective of Temporal State Transport. We introduce Spectral Tension, a signed diagnostic that compares local attention diffuseness with global spectral diversity, and use it to identify two opposite temporal failures: fragmented transport and over-mixing hotspots. Based on this diagnosis, we propose Spectral Transport Homeostasis, a training-free regulator that softly corrects pathological temporal states while largely preserving balanced ones. Experiments on pretrained video generation models show that the original model often occupies imbalanced temporal regimes, whereas our method selectively applies larger corrections to the worst temporal hotspots and improves temporal consistency and visual quality without finetuning. Code: https://github.com/lytang63/temporal-state-transport