Video generators shape motion using geometry but ignore physical laws
How Does Geometry Enter Generated Motion?
Computer Vision and Pattern RecognitionArtificial Intelligence
Summary
This paper studies how AI-generated videos use the picture’s shape to decide how things move, focusing on balls rolling on tracks. The authors found that while the ball’s motion roughly follows the track’s shape, the videos don’t strictly follow real physical laws—like a ball speeding up or climbing too high unrealistically. They also found that text prompts largely steer where the ball ends up, and how long the video is controls timing, rather than physics. This means current video AI acts more like a motion imitator, not a true physics simulator.
What this means in practice
- •For video game developers: Simulate ball motion on virtual tracks that respond visually to track shape changes without full physics simulation.
- •For visual effects artists: Create plausible animation of rolling objects on complex surfaces shaped by the initial image geometry and text prompts.
Authors
Weihan Li, Junhao Wu, Yuhan Song, Xiaofeng Lin, Xinlei Chen
Abstract
Under a fixed physical law, the visible geometry of a scene determines how motion must change. We ask how video generators realize this relationship. We fix the law and the initial state and change only the geometry drawn in the first frame, within matched families of tracks and deflectors, and compare each generated trajectory with the simulator prediction for that geometry. Paired interventions change one thing at a time: a local bump, the height of a barrier, the words of the prompt, the length of the clip. Across nine image-to-video models, geometry is preserved and shapes the motion: the speed of the ball follows the drawn undulation of a track. A physical state would carry this response forward, and here the generated motion parts from the law. The mean slope barely accelerates the ball, successive contacts fail to compose through a consistent state, an edit ahead of the ball alters its motion before it arrives, and the ball climbs over barriers higher than its release point. Two global conditions organize the global trajectory: text strongly controls the destination, while clip length strongly controls timing in the open-weight models tested. The pattern persists with photographed first frames. Current video generation thus behaves as geometry-conditioned motion synthesis whose evolution of state differs systematically from that of a fixed physical law.