WorldAlign improves video generation with better static and dynamic consistency
WorldAlign: Decoupled 4D Reward for World-Consistent Video Generation
Computer Vision and Pattern Recognition
Summary
Making videos that look realistic means making sure everything stays consistent over time and from different angles. The authors created WorldAlign, a new way to check that both the still parts of a scene and moving subjects behave correctly in generated videos. They separate the static background from moving objects and use special models to judge each part’s accuracy. This approach helps improve video quality without extra human feedback and works well on existing video generation tools.
What this means in practice
- •For video game developers: Improve the realism of dynamic scenes by ensuring consistent backgrounds and motion in generated game video content.
- •For visual effects teams: Enhance post-production video generation by better aligning moving subjects and static scene elements for consistent rendering over time.
Authors
Jing He, Kaixin Ding, Xingye Tian, Guibao Shen, Wenhang Ge, Xin Tao, Pengfei Wan, Ying-Cong Chen
Abstract
Faithful visual world simulation requires generated videos to maintain 4D world consistency, encompassing both static and dynamic consistency. Static consistency requires coherent 3D structure in static environments across viewpoints, while dynamic consistency requires plausible subject motion and consistent appearance over time. Geometry-aware post-training offers a promising way to improve world consistency. However, existing methods often rely on a static-scene assumption. Even those that accommodate dynamic scenes struggle to provide reliable static-consistency feedback, while dynamic consistency is often overlooked or inadequately assessed. To address these limitations, we introduce WorldAlign, a decoupled 4D reward framework that semantically separates static regions and dynamic subjects and provides feedback by aligning each with a world prior suited to its assumptions. For static regions, WorldAlign aligns static geometry with a geometric world prior through semantically guided masked reprojection, enabling more reliable static-consistency evaluation; an auxiliary camera-motion reward discourages nearly static solutions. For dynamic subjects, WorldAlign uses a strong vision-language model (VLM) as a dynamic world prior and constructs a VLM-as-a-judge reward based on sample-specific checklists that assess dynamicity, physical plausibility, shape, and texture consistency. This decoupled design enables more effective online post-training without requiring human preference annotations. Across two pretrained image-to-video generators, Wan2.1 and Wan2.2, WorldAlign jointly improves static and dynamic consistency over existing methods without suppressing overall or subject motion. These results support decoupled world-prior alignment for more faithful visual world simulation. Project page: https://worldalign.github.io/.