Velox speeds up video simulation with memory and geometry tricks
Velox: Spatial Memory Makes World Models Efficient
Computer Vision and Pattern Recognition
Summary
Video world models simulate environments by predicting what happens next from actions and camera views, but they slow down as they remember more past video frames. The authors introduce Velox, a system that uses 3D camera views and depth information to smartly select which parts of past video to pay attention to, making the simulation faster without losing quality. This approach cuts down on the usual slowdown, keeping the process responsive and accurate over longer video sequences. Velox runs efficiently on modern hardware and achieves major speed gains while maintaining similar performance to existing methods.
What this means in practice
- •For video game developers: Generate longer interactive visual scenes more quickly by improving memory management during video-based simulations.
- •For robotics software teams: Support real-time navigation simulations using efficient video world models that keep relevant spatial memory without slowdown.
Authors
Qiuhong Shen, Shizun Wang, Xinchao Wang
Abstract
Video world models turn actions and camera motion into long visual rollouts, making video a natural interface for interactive simulation. Such interaction requires responsive generation, yet coherent rollouts must retain visual memory of what has already been generated. Chunked video diffusion models usually keep this context in growing KV caches, so later chunks become progressively slower. To address this bottleneck, we introduce Velox, a training-free attention router for region-level context selection using frustum slab correspondences from streaming geometry. Velox represents historical blocks as camera frustum slabs with depth intervals and obtains current-block depth through recency-priority multiwarp. For well-covered regions, frustum slab overlap softly modulates each layer's model-derived block importance. Uncovered regions retain the model-derived route without geometric bias. The fixed token-block granularity matches FlashAttention-style tiled execution and reduces the attention term that grows with rollout length. Frustum slab construction runs asynchronously with denoising on the same H200. At chunk 48, Velox accelerates attention by $4.36\times$ and denoising by $2.43\times$. On the full 158-case WBench navigation split, it remains within 0.15 API-free average points of dense LingBot-Fast-Cam. Please see our project page at https://velox-world.com/velox/ for more details.