OneWorld improves consistent physics understanding across actions in video prediction
OneWorld: Learning Consistent Physics Across Actions in World Models
Computer Vision and Pattern RecognitionArtificial Intelligence
Summary
Predicting how a scene changes after different actions is hard because each prediction might assume different physical rules, making it confusing. The authors propose OneWorld, a method that learns a shared physical explanation behind various possible outcomes for different actions. This makes predictions agree on basic physics like friction or mass, helping the model better understand and plan in dynamic environments. They also create a way to test if the predictions use consistent physics across multiple actions. Experiments show that OneWorld keeps good prediction quality while improving physical consistency across scenarios.
What this means in practice
- •For robotics engineers: Create robots that anticipate consistent physical effects of different actions to improve planning and interaction in changing environments.
- •For simulation software developers: Build video simulation tools that generate physically coherent scenarios under multiple action choices for training and evaluation.
Authors
Ke He, Yichen Ding, Bin Yang
Abstract
Action-conditioned video world models aim to predict scene evolution under different actions, a capability that is essential for reliable planning, decision-making, and interaction in dynamic environments. However, futures generated independently from the same initial scene may each appear plausible while implying incompatible physical properties, such as friction or mass. This inconsistency can lead to contradictory predictions across interventions, making it difficult for the model to maintain a coherent understanding of the underlying world and limiting its reliability for planning and decision-making. To address these issues, we propose OneWorld, a shared-mechanism counterfactual generation framework that jointly models multiple action-conditioned futures under a common latent physical mechanism. A physical mechanism interpreter first infers a distribution over latent mechanisms from each action-outcome branch. These distributions are then aggregated into shared-world evidence, which captures whether the branches admit a common physical explanation while accounting for uncertainty in less informative branches. This evidence constrains flow training and guides sampling, encouraging consistency in the underlying physical mechanism while preserving the distinct outcomes induced by different actions. We further introduce a multi-intervention evaluation protocol in controlled environments, following the interaction settings of ACWM-Phys, to assess whether generated futures can be jointly explained by the same physical parameters, alongside standard measures of single-rollout prediction quality. Experiments in these environments show that OneWorld improves cross-intervention physical consistency while maintaining competitive single-rollout prediction quality.