Actionsplice enables instant action updates in video world models
ActionSplice: In-Flight Action Editing for Interactive World Models
Computer Vision and Pattern RecognitionMachine Learning
Summary
When making video models that predict future scenes based on actions, updating an action partway through is tricky and often slow. The authors introduce ActionSplice, a method that lets these models quickly adjust to new actions without redoing all the previous work. It uses a lightweight corrector to shift the model’s state to match the new action and allows smooth continuation. This approach improves accuracy and speeds up video prediction when actions change during generation.
What this means in practice
- •For game developers: Update in-game predicted scenes immediately when player actions change, improving interactivity without slowing down gameplay.
- •For robotics engineers: Adjust robot world models on-the-fly with new control commands, enabling smoother task adaptation without full model recomputation.
Authors
Pardis Taghavi, Tingyu Guo, Jonas Lossner, Gaurav Pandey, Reza Langari
Abstract
Chunk-autoregressive video world models typically condition each generated chunk on one action. An action received during sampling must therefore wait for the next chunk, condition future solver evaluations on a state produced under the previous action, or trigger rollback that repeats completed evaluations. We introduce ActionSplice, an inference framework that formulates this problem as Counterfactual State Transport (CST). A lightweight corrector transports the interrupted backbone-native representation toward the matched state induced by the revised action at the same solver step. The world model and sampler remain frozen, and sampling resumes without replaying completed evaluations. The retargeting variant $\mathrm{CST}*{R}$ updates the entire active chunk, while the temporal-splicing variant $\mathrm{CST}*{T}$ preserves a temporal prefix and updates only the suffix. Across minWM-Wan Action2V and HY-WM1.5, $\mathrm{CST}*{R}$ reduces rollback-relative LPIPS by 61.5% and 75.9% relative to direct condition swapping. $\mathrm{CST}*{T}$ reduces suffix LPIPS by 56.1% and 77.5%, respectively, while providing $2.73\times$ and $1.69\times$ pixel-ready speedups over waiting. Under the HY-WorldPlay protocol, $\mathrm{CST}_{R}$ obtains a PSNR of 25.66 dB, an SSIM of 0.6902, and an LPIPS of 0.1337 against the original rollout.