EditWorld enables precise video editing in interactive virtual worlds
Precise Editing and Flexible Referencing for Interactable Worlds
Computer Vision and Pattern RecognitionArtificial Intelligence
Summary
It’s hard to change things exactly in virtual video worlds once they’re created. The team behind EditWorld made a method that lets users carefully edit what happens in these interactive video scenes as they play out. Their system can take editing instructions and pictures to guide changes over time, keeping track of past events without needing too much memory. They also made a way to measure how well these edits work and showed that EditWorld outperforms earlier methods.
What this means in practice
- •For game developers: Enable real-time precise modification of virtual worlds during gameplay for customized experiences.$Commercial implications: Allows creation of interactive games with user-driven video world edits enhancing immersion and storytelling.
- •For animation studios: Edit complex animated video scenes flexibly using streamed instructions and reference images to refine content efficiently.
Authors
Xinyao Liao, Xianfang Zeng, Zhu Liang, Zhoujie Fu, Qianxun Xu, Jiachi Liu, Gang Yu, Guosheng Lin
Abstract
We present EditWorld, a video world model for precise editing and flexible referencing in interactable worlds. Existing video world models primarily focus on navigation, letting users explore generated worlds but offering limited control over how existing world content is modified. EditWorld extends world modeling from exploration to precise modification by streaming editing instructions and reference images during autoregressive generation. To support these capabilities, EditWorld introduces Gated Causal Attention for temporally varying editing conditions and reference images, together with a Sparse Context mechanism that maintains a bounded historical context for long-horizon inference. We further adopt joint autoregressive and bidirectional training with annealed self-resampling, and construct a dedicated data synthesis and annotation pipeline that provides supervision for world editing. We also present WBench-Editing to systematically evaluate streaming world editing capabilities. EditWorld achieves the best overall performance on WBench-Editing with an overall score of 73.8 and an editing score of 80.0, substantially outperforming existing methods on editing-related metrics. https://github.com/leoisufa/EditWorld