Correcting a learned physical invariant improves world-model rollouts
2026-08-24 • Artificial Intelligence
Artificial Intelligence
AI summaryⓘ
The authors studied a video prediction model called DreamerV3 that was trained only on videos of a swinging pendulum. They found that the model learns something like an energy value that stays almost the same during its predictions, but only in models without friction. If they force the model’s internal state to keep this energy constant, its predictions get better. However, when the model imagines forward, this energy drifts, showing that even though the model sees physical rules in the video, it doesn't perfectly follow them when predicting ahead.
world modelsDreamerV3latent spaceenergy invariantpendulumvideo predictiondynamicsrolloutsphysical constraintslatent transition
Authors
Richard Bao
Abstract
World models can predict video without learning dynamics that they reliably preserve. We test whether a frozen DreamerV3 trained only on pendulum video learns a scalar that its own latent transition treats as approximately conserved. A label-free search recovers the same energy-like invariant across independently trained conservative models, while the same procedure finds no comparable invariant in matched damped models. During autonomous rollouts, this quantity drifts. Projecting the latent state back toward its initial level set reduces rollout error in all three conservative models, whereas matched random constraints usually increase it. These results distinguish a dynamically meaningful invariant from a merely decodable correlate and reveal a concrete failure mode: a world model can learn a physical constraint from pixels yet violate that constraint when it imagines forward.