Rate adaptive rollout improves flexibility of video interactive world models
RAR: Rate-Adaptive Rollout for Time-Consistent Interactive World Models
Computer Vision and Pattern Recognition
Summary
Interactive video world models let users control virtual environments, but they usually work at a fixed speed, which limits how smoothly the virtual world can run or how far into the future it can show. The authors introduce Rate-Adaptive Rollout (RAR), which lets these models work at different speeds while keeping the timing consistent and without losing track of past information. Their method uses a special way to line up video frames from different speeds and trains the model to handle both detailed motion and long-term changes. This helps virtual worlds adapt in real time and show longer-lasting views without losing quality.
What this means in practice
- •For game developers: Create video-based interactive environments that adapt runtime speed without losing quality or consistency.$Commercial implications: This enables smoother and more flexible virtual worlds for games, improving user experience and control.
- •For simulation engineers: Deploy adaptive-rate rollout models to extend simulation horizons and enhance temporal accuracy under fixed compute budgets.
Authors
Yaoyang Liu, Dongyu Yan, Yihua Du, Shengju Qian, Zeke Xie, Xin Wang, Ying-Cong Chen, Long Chen
Abstract
Video-based interactive world models aim to enable real-time, user-controllable environments. Yet most existing systems are trained and rolled out at a fixed temporal rate, limiting both runtime-rate adaptation and physical-horizon coverage -- a limitation we term temporal-rate coupling. Specifically, when runtime generation throughput differs from the learned temporal rate, the generated world evolves unintentionally more slowly or quickly. Moreover, using a single temporal rate imposes an inherent trade-off between temporal granularity and physical horizon: higher rates capture finer motion but cover a shorter duration under a fixed context window. To address both limitations, we introduce Rate-Adaptive Rollout (RAR), a framework that enables interactive world models to roll out at controllable temporal rates. At its core, Stride-RoPE aligns latent frames sampled at different rates on a shared physical-time coordinate, enabling consistent multi-rate rollout, unseen-rate generalization, and online rate switching without discarding history. Meanwhile, RAR uses multi-stride video sampling to enable joint learning of fine-grained dynamics and long-horizon evolution. Across action- and camera-controlled environments, RAR achieves temporally consistent multi-rate rollout and online rate switching while preserving high-rate generation quality and extending physical horizons under matched training budgets. RAR also transfers to pretrained world models while preserving perceptual quality.