Actionsplice speeds up editing of video world models during generation
ActionSplice: In-Flight Action Editing for Interactive World Models
Computer Vision and Pattern RecognitionMachine Learning
Summary
Video world models predict future scenes based on actions, but changing an action partway requires restarting or waiting. The authors propose ActionSplice, a method that quickly updates the video prediction when actions change by adjusting model states without redoing all calculations. This approach is faster and produces better quality updates compared to previous methods that had to replay or wait. It works by transporting internal model representations to match the new action's effects in real time.
video world modelschunk-autoregressiveaction conditioningcounterfactual state transportrollbacksamplingLPIPSPSNRSSIMinference framework
Authors
Pardis Taghavi, Tingyu Guo, Jonas Lossner, Gaurav Pandey, Reza Langari
Abstract
Chunk-autoregressive video world models typically condition each generated chunk on one action. An action received during sampling must therefore wait for the next chunk, condition future solver evaluations on a state produced under the previous action, or trigger rollback that repeats completed evaluations. We introduce ActionSplice, an inference framework that formulates this problem as Counterfactual State Transport (CST). A lightweight corrector transports the interrupted backbone-native representation toward the matched state induced by the revised action at the same solver step. The world model and sampler remain frozen, and sampling resumes without replaying completed evaluations. The retargeting variant $\mathrm{CST}*{R}$ updates the entire active chunk, while the temporal-splicing variant $\mathrm{CST}*{T}$ preserves a temporal prefix and updates only the suffix. Across minWM-Wan Action2V and HY-WM1.5, $\mathrm{CST}*{R}$ reduces rollback-relative LPIPS by 61.5% and 75.9% relative to direct condition swapping. $\mathrm{CST}*{T}$ reduces suffix LPIPS by 56.1% and 77.5%, respectively, while providing $2.73\times$ and $1.69\times$ pixel-ready speedups over waiting. Under the HY-WorldPlay protocol, $\mathrm{CST}_{R}$ obtains a PSNR of 25.66 dB, an SSIM of 0.6902, and an LPIPS of 0.1337 against the original rollout.