World models improve video view changes with trained-free control
World in World: Explore the World with World Models
Computer Vision and Pattern Recognition
Summary
Changing the viewpoint of a video while keeping everything looking real and consistent is very hard. The authors created a method called World in World that lets an existing video model handle different camera angles and times without needing extra training. Their method combines clues from the original video, 3D geometry, and previous video states to fill in new views smoothly and keep the scene realistic. This helps create videos where you can change the camera angle or revisit scenes over a long time and still see believable and consistent content.
What this means in practice
- •For video game developers: Enable real-time camera angle changes in game replays or cutscenes using existing video data without retraining models.
- •For special effects artists: Create realistic video effects that allow flexible viewpoint changes and motion transfers on source footage without additional model training.
Authors
Chenxi Song, Yanming Yang, Chi Zhang
Abstract
Autoregressive video world models enable interactive, long-horizon exploration, but flexible control remains challenging. Exploring a source video from new viewpoints requires the generated rollout to remain synchronised with the recorded event, place observed content in the requested view, plausibly complete newly exposed regions, and recover previously generated appearance on revisits. Existing methods typically address these requirements through task-specific modules or additional training. We present World in World, a training-free inference-time interface that converts heterogeneous control evidence into camera- and time-labelled clean visual states, which are read through the native self attention of a frozen causal video model. The evidence comprises source-video observations, target-view scene projections, geometry renderings that guide completion of newly exposed subject regions, and retrieved generated states beyond the rolling cache. Each evidence source carries token-level support and its own availability schedule. A correspondence router combines persistent point identities with geometry to establish token correspondences, guiding supported queries towards matching source-video tokens. Evidence-wise attention CFG (EWA) then independently regulates each auxiliary channel's additional contribution using attention responses from the same denoising forward pass. The shared interface supports camera-controlled rerendering, long-horizon revisiting, and human-motion transfer with the same frozen backbone. We evaluate World in World on camera-controlled video rerendering under diverse viewpoint changes, assessing perceptual quality, temporal consistency, and camera-following accuracy.