Vision language delta enables robot task state changes with compact models
Vision-Language-Delta-Action: Learning Compact Deltas as Actionable Transition Interfaces for Robotic Manipulation
RoboticsComputer Vision and Pattern Recognition
Summary
Robots need to change the world around them to complete tasks, not just react with actions from what they see. The authors introduce a new way to help robots by focusing on compact, important changes in the state of the world, rather than predicting every detail of the future. Their approach breaks down state changes into meaningful pieces called Deltas that guide the robot's actions better. This method improves how well robots plan and act over longer tasks and in new situations.
What this means in practice
- •For robotics engineers: Build robot control systems that reliably execute complex tasks by predicting key state changes rather than full future states.
- •For warehouse automation teams: Improve long-term planning and robustness of manipulation robots handling diverse objects and tasks through compact actionable models.$Commercial implications: Enables deployment of robots with better task consistency and adaptability, providing a competitive operational advantage.
Authors
Lingsen Zhang, Wei Li, Jie He, Henghua Zhang, Kai Mo, Shiyong Li, Yanpeng Wang, Rui Shao
Abstract
Robotic manipulation is not merely observation-to-action generation, but the realization of task-specified state changes in the physical world. Existing vision-language-action (VLA) methods directly learn observation-to-action mappings, imposing a substantial learning burden due to the gap between semantic understanding and action execution. Although world-action models (WAMs) improve control through future prediction, modeling complete future states wastes capacity on static appearance and action-irrelevant changes. We propose Vision-Language-Delta-Action (VLDA), a paradigm that reformulates full future prediction as compact actionable transition modeling. Its core is a compact Delta variable that captures latent world-state changes, serving as an interface between vision-language understanding and action generation. VLDA learns Delta through a unified objective composed of the Local Transition Filter (LT-Filter) and Global Transition Anchor (GT-Anchor). LT-Filter captures task-relevant manipulation-centric changes while suppressing static reconstruction biases. GT-Anchor provides coarse future-level regularization to preserve the overall direction of state changes. Delta-Driven Action Refinement (DDAR) further grounds Delta in the action manifold and injects it as a structured seed to guide action denoising. We implement VLDA within LoongForge, an open-source training framework for large-scale foundation and embodied models, using its modular and distributed infrastructure for unified two-stage optimization. Experiments show that VLDA improves long-horizon consistency, robustness, and generalization, validating actionable Delta as an alternative to direct action prediction and full future-state modeling.