WorldCast evaluates AI models predicting future visual world states
WorldCast: Exploring Intelligent Visual Generation in World State Prediction
Computer Vision and Pattern RecognitionArtificial Intelligence
Summary
Predicting what the world will look like next is hard because it requires understanding many things at once, like actions, changes, and reasoning. The authors created WorldCast, a set of test cases to see how well different AI models can predict future visual scenes from multiple kinds of information. Their tests show that some advanced AI systems are starting to get better at this, but many still struggle with complex situations. The study highlights the need for AI that can think ahead, plan, and combine different types of knowledge to better imagine what comes next.
What this means in practice
- •For robotics teams: Improve robots’ ability to predict and visualize future environments from combined sensors and instructions for better decision making.
- •For game developers: Enhance non-player character behaviors by forecasting future world states visually from complex player actions and game contexts.
Authors
Wenhao Yan, Haodong Li, Xiaotong Li, Yaxin Luo, Zhengrong Yue, Jialv Zou, Xu Huang, Haojin Yang, Nan Jia, Yan Feng, Peng Pei, Xunliang Cai
Abstract
Humans interpret multimodal observations of the world and forecast how it will evolve. Recent multimodal generative systems have demonstrated similar capabilities: they can integrate multimodal context and synthesize visual outputs that depict changes in the world. To characterize this emerging trajectory, we organize model capabilities into a four-level hierarchy and discover that higher-level tasks remain relatively underexplored despite model progress. To bridge this gap, we explore model performance on multi-step world state prediction, a high-level task that requires models to forecast successive future world states from multimodal observations and diverse references. Such a task requires models to possess entangled understanding and generation capabilities, as well as extensive world knowledge to translate human action sequences, physical transformations, and abstract visual reasoning into coherent visual outputs. We introduce WorldCast, a benchmark comprising 320 manually designed cases across eight categories, and evaluate 12 representative models. The conducted experiments yield three key insights: (a) advances in frontier models are rapidly pushing the boundaries of intelligent world modeling, with the GPT-Image-2.5 System scoring 72.8 and demonstrating early-stage visual state prediction capabilities on real-world tasks; (b) WorldCast reveals substantial gaps between models: even Nano Banana Pro reaches only 42.8 and the best open-source model scores 20.1, with deficits in handling complex context, entangled references, and reasoning-heavy tasks; (c) strong generative models effectively draw on the capabilities of their underlying language models. These findings highlight the need for multimodal generative models that are capable of dynamic planning and reasoning while also being deeply unified to faithfully simulate real-world states.