ScaleWorld enables efficient high-resolution video world generation
ScaleWorld: High-Resolution World Generation through Shared Prediction and Refinement
Computer Vision and Pattern Recognition
Summary
Generating detailed, high-resolution videos of virtual worlds usually takes a lot of time and computer power. The authors propose ScaleWorld, a model that first creates a low-resolution video prediction and then improves it within the same system. This approach makes it possible to produce detailed videos without needing separate models or huge resources. They also introduce a special training method to help the parts of the model work well together. Their experiments show the method is both efficient and produces high-quality video generation.
What this means in practice
- •For game developers: Generate detailed in-game video environments efficiently by integrating low- and high-resolution video prediction into one model.
- •For augmented reality engineers: Create realistic streaming virtual scenes with lower computing costs using a single model for prediction and refinement.
Authors
Yansong Qu, Wesker Zheng, Quanjian Song, Zhiyuan Zhang, Xiao Cao, Shengchuan Zhang, Liujuan Cao
Abstract
High-resolution generation enables video world models to render fine-grained scene structures and textures, but directly generating high-resolution video incurs substantial computational and memory costs. We present ScaleWorld, a causal video model that unifies low-resolution world prediction and high-resolution refinement within a single shared backbone. By refining its own predictions, ScaleWorld produces high-resolution world videos without requiring a separate refinement model. For each video chunk, the shared backbone first completes a low-resolution prediction and then performs one-step high-resolution refinement conditioned on that prediction. To support efficient streaming generation, we maintain separate causal states with a long low-resolution history for prediction and a short high-resolution history for refinement. To address task interference during distillation, we separate specialist optimization from capability consolidation and introduce task-asymmetric on-policy distillation (TA-OPD). This procedure transfers prediction and refinement behaviors from specialized teachers into a shared student through local velocity matching under their respective sampling and history policies. {Extensive experiments demonstrate that ScaleWorld achieves high-quality world generation with improved computational and memory efficiency.