Code based system builds persistent worlds with video visuals
Code Plans, Diffusion Renders: Open-Ended Generative World Modeling
Computer Vision and Pattern Recognition
Summary
Many computer models that simulate the world only use images and videos to show what happens, but this new system called CoDeR actually writes and runs code to create a world that can keep evolving. The authors combine several parts that help turn simple ideas into detailed rules, actions, and visuals you can see. Because it uses code, the world can remember things for a long time and let multiple characters interact and change over time beyond what is immediately visible. Their experiments show it works better than older models in keeping complex, long-term scenes and interactions.
What this means in practice
- •For game developers: Create persistent game worlds where multiple characters can continuously interact and evolve with realistic visual scenes.$Commercial implications: Enables next-generation video game engines that embed executable world rules for dynamic and long-term gameplay.
- •For robotics simulation teams: Simulate complex environments where multiple autonomous robots interact and evolve over extended scenarios with realistic visuals.
Authors
Zixun Fang, Yawen Shao, Kai Zhu, Jie Xiao, Shihan Chen, Yu Liu, Xueyang Fu, Yang Cao, Wei Zhai, Zheng-Jun Zha
Abstract
We introduce \textbf{CoDeR}, a new paradigm for world modeling. Unlike existing video world models that implicitly represent world dynamics through visual observations, our system explicitly constructs an executable world with code and employs video generation models for visual realization. Specifically, we coordinate five complementary roles to translate high-level concepts into structured world rules, executable dynamics, and perceptual observations. This design enables \textit{long-term memory}, \textit{open-ended interactions}, \textit{autonomous world evolution}, and \textit{multi-agent scenarios}, where multiple entities can act, interact, and evolve persistently beyond the current observation. Extensive experiments demonstrate that our framework substantially extends the capabilities of existing world models, enabling long-term memory, open-ended interactions, autonomous evolution, and persistent multi-agent dynamics, while achieving state-of-the-art performance across multiple evaluation settings. Code and model weights will be made publicly available. Project Page: \href{https://becauseimbatman0.github.io/CoDeR}{CoDeR}.