FlowVLA improves robot manipulation with 3D motion learning
FlowVLA: Enabling World Dynamics Understanding through Pluggable 3D Flow Learning
Robotics
Summary
Robots today mainly learn from flat pictures and text, which limits how well they understand moving things in 3D space. The authors created FlowVLA, a way to teach robots about the real 3D movements around them by learning from 3D flow during training only. This method helps robots predict how objects and their own arms move physically in the world without needing extra 3D sensors or slower processing when working. Testing shows FlowVLA helps robots handle tasks better, understand changes they’ve never seen before, and learn more efficiently.
What this means in practice
- •For robotics engineers: Improve robot arm manipulation accuracy by integrating 3D motion learning during training without extra 3D sensors or runtime cost.
- •For autonomous system developers: Enhance robotic system robustness to new tasks and environments by training with dynamic 3D flow data while maintaining efficient 2D model inference.
Authors
Junhyeong Go, Mingyu Jung, Woobin Im, Jongbin Ryu
Abstract
Recent advances in 3D reconstruction enable the extraction of temporally consistent 3D motion from RGB observations; however, current Vision-Language-Action (VLA) models still predominantly rely on 2D image-text priors. Therefore, they do not accurately represent the fundamental dynamics of the 3D world associated with physical actions. This over-reliance on static 2D visual cues limits the capacity of VLA models in achieving robust robot manipulation. To address these limitations, we introduce FlowVLA, a reconstruction-guided, pluggable 3D flow learning framework aimed at empowering VLA models with an inherent understanding of action-relevant dynamics. Instead of requiring explicit 3D sensor inputs or altering the inference-time architecture, our method is designed to utilize temporally aligned 3D flow only during training. This allows the model to learn the dynamic evolution of the physical scene while preserving the computational efficiency of conventional 2D VLA backbones. To bridge scene-level dynamics with robotic manipulation, we propose a modular 3D flow learning framework with dual-level flow supervision. Specifically, point-level flow guides the model to forecast the future displacements of task-critical 3D points, while TCP-level flow aggregates these motions to align with the robot's TCP trajectories. This dual-level flow supervision encourages the model to focus on semantically meaningful dynamic regions, effectively coupling scene-level motion with precise control signals. Extensive evaluations across simulation and real-world experiments demonstrate that FlowVLA consistently improves manipulation performance, zero-shot robustness, and optimizes data efficiency. FlowVLA adds only lightweight auxiliary heads during training and discards them at inference. Therefore, FlowVLA introduces no additional 3D sensor inputs, architectural modifications, or inference-time computation.