Agentic video world model builds persistent virtual worlds from images
AgiWorld: An Agentic Video World Model for Building Persistent Worlds
Computer Vision and Pattern RecognitionArtificial Intelligence
Summary
Making videos that let you explore a virtual world from just one picture is tricky because the world often doesn’t stay the same as you move around. The authors created AgiWorld, which uses an intelligent agent to plan the journey and build the world step-by-step, remembering what it has seen so far. This approach helps keep the world consistent and faithful to instructions even on longer explorations. The video model and the agent work together to fix problems like parts of the view being hidden or errors building up over time.
What this means in practice
- •For game developers: Create virtual environments that persist and evolve realistically from minimal visual input and user instructions.
- •For virtual reality content creators: Generate faithful and stable immersive scenes automatically from a single image with higher-level control over exploration paths.
Authors
Shaoheng Fang, Yiqun Mei, Hanwen Jiang, Xin Sun, Haitian Zheng, Hao Tan, Kalyan Sunkavalli, Georgios Pavlakos, Qixing Huang, Yiwei Hu
Abstract
Recent interactive video world models generate visually compelling videos from a single image, following user-provided camera motion and text prompts. However, such low-level control demands substantial user effort to realize a high-level intent, and the generated world struggles to remain persistent as the exploration proceeds. In this work, we present AgiWorld, an agentic video world model that explores and builds a persistent world from a single image and a free-form instruction. For high-level planning, our agent AgiWorld-Agent designs the world and the journey through it in a world state and directs the exploration. For low-level realization, our video world model AgiWorld-WM generates the video chunk by chunk and stores it with its geometry in a spatial memory, from which the agent updates its world state. This closed loop keeps the world faithful to the user's intent and keeps the agent's world state consistent with the generated content throughout the exploration. We co-design the two levels around the shared spatial memory and an interface of metric camera trajectories and structured prompts. In AgiWorld-WM, to handle occlusion and accumulated geometric errors in the spatial memory over long rollouts, we propose occlusion-aware history selection, which retrieves past views by surface visibility, and reliability-based memory maintenance, which prefers reliable geometry and earlier observations. AgiWorld-WM achieves state-of-the-art overall performance on world revisiting and exploration benchmarks, and qualitative results and ablations show that AgiWorld builds persistent worlds from free-form instructions over minute-long journeys. Persistence thus arises not from the video model alone but from its closed loop with the agent.