PAWBench: How Far Are We from Probabilistically Aligned World Modeling?
Computer Vision and Pattern RecognitionArtificial Intelligence
Summary
The authors study video generation models that try to predict how the physical world changes over time, called world models. They argue that these models should not only create one possible future video but also capture all the different ways the scene could realistically unfold, which they call probabilistic alignment. To test this, they made a benchmark called PAWBench and a method called PAWEval to see if models generate a correct range of possible behaviors from the same start. After testing 11 models, the authors found none consistently matched the true variety of possible outcomes. They also explored if changing prompts, noise, or training helped, showing that there is still work needed for better probabilistic world models.
video generationworld modelsprobabilistic alignmentdistributional criterionstochastic samplingbenchmarkvideo rolloutsphysical behaviorsmodel evaluationpredictive distribution
Authors
Yuandong Pu, Le Zhuo, Sayak Paul, Gabriel Jorge Menezes, Avram Đorđević, Shiyang Li, Yifan Zhou, Bin Fu, Wenlong Zhang, Junjun He, Yu Qiao, Yihao Liu, Jingbo Xing, Xi Chen
Abstract
Recent video generation models are increasingly framed as world models. Many physical processes can unfold in more than one valid way. Therefore, a world model should reproduce not only a plausible trajectory, but also the distribution of possible behaviors under the same initial observation and action. We call this distribution-level requirement probabilistic alignment. However, existing evaluations largely assess individual-video plausibility and do not test whether repeated generations recover the correct distribution. This raises a central question: how far are current video generators from probabilistically aligned world modeling? To answer it, we formalize probabilistic alignment as a distributional criterion for world models and introduce PAWBench, a benchmark for evaluating video generators as stochastic samplers of world dynamics. We further introduce PAWEval, an outcome-level protocol that converts repeated video rollouts into empirical distributions over possible physical behaviors. Across 50 scenarios and eleven current systems, no model consistently matches the reference probabilities while recovering the range of valid behaviors. Having established this gap, we test whether language prompts, initial noise sampling, or model training can reshape the model's predictive distribution. We believe our work can serve as a foundation for future efforts to move towards probabilistically aligned world modeling.