Distilling robotic video prediction models into one-step fast control
DIDO: Distilling Interaction-Centric Dynamics into One-Step Denoising for World Action Models
Robotics
Summary
Robots need to predict what will happen next to handle objects well, but current video models take many steps to create each prediction, making them slow. The authors found that important moving parts, like the robot’s gripper and object, become clear only after several steps, not right away. They developed DIDO, a way to teach a fast one-step model to capture these crucial interactions accurately. This new method makes robot control quicker without losing the ability to understand how objects will move and interact.
What this means in practice
- •For robotic manipulation engineers: Speed up robot grasping and object handling by using one-step prediction models that keep important interaction details.
- •For autonomous system developers: Improve real-time decision making in robots by reducing prediction latency while preserving accuracy in object interactions.
Authors
Jing Lyu, Shuanghao Bai, Runze Xiao, Zhenyu Liao, Wenxing Tan, Zihan Tang, Ruochuan Shi, Cheng Peng, Yuheng Ji, Yihao Wang, Badong Chen, Pengwei Wang, Zhongyuan Wang, Xiaoguang Zhao
Abstract
World Action Models (WAMs) use video generation models to predict future visual dynamics for robotic manipulation, but iterative denoising introduces additional latency for closed-loop control. We empirically find that visual content converges at different rates during denoising. Static background structure forms early, whereas the gripper and manipulated object remain blurry after the first step, with their interaction dynamics emerging only through subsequent denoising. Consequently, naively truncating a multi-step video model to one step preserves scene structure but loses the interaction-centric dynamics most critical for manipulation. To address this issue, we propose DIDO, which distills the converged dynamics of a multi-step video model into a single denoising step. DIDO combines distribution matching distillation with interaction-centric representation guidance. Beyond compressing multi-step generation into one forward pass, DIDO explicitly models the gripper, manipulated object, and their interaction using supervised bounding-box visual reasoning tokens. Additionally, DIDO aligns the target object's representations across multiple model layers with features from a pretrained DINOv3 encoder. This interaction-centric guidance helps the distilled model preserve both the relevant entities and their future dynamics in a single step, while substantially reducing inference latency. DIDO achieves an average success rate of 99.0\% on LIBERO, 76.6\% on LIBERO-Plus, and 92.0\% on RoboTwin, while also demonstrating effective transfer to long-horizon and generalization tasks in real-world robotic manipulation.