VFlash speeds up visual planning for robots with more accuracy
VFlash: Fast, Accurate Visual Planning via Flash Primitive Decoding
RoboticsComputer Vision and Pattern Recognition
Summary
Robots and self-driving cars need to plan where to move by understanding visual goals, but current methods are slow because they predict one point at a time. The authors present VFlash, a technique that predicts whole shapes or paths all at once, making planning faster and more precise. It also accepts varied valid predictions instead of strict ones, improving flexibility. Tests show it reduces errors and speeds up predictions by about 25 times in 3D tasks, helping robots complete tasks more successfully.
What this means in practice
- •For robotics engineers: Generate accurate visual motion plans quickly to improve autonomous robot control and task success.
- •For autonomous vehicle developers: Improve prediction speed and precision of spatial goals for vehicle motion forecasting and navigation.
Authors
Yu-Cheng Chou, Shihao Wang, Jin Huang, Chi-Pin Huang, Chiao-An Yang, Xianzhe Fan, Jing Wang, Zhiqi Li, Yuanguo Kuang, Ruijie Zheng, Johan Bjorck, Sifei Liu, Hang Su, Abhishek Badki, Lei Zhang, Alan Yuille, Linxi "Jim'' Fan, Jan Kautz, Zhiding Yu
Abstract
Models for physical AI tasks, such as robotics and autonomous driving, need not reason solely in language. They can use visual plans--points, bounding boxes, and trajectories--to represent spatial goals and anticipate motion. However, autoregressive vision-language models generate these plans one coordinate token at a time, making richer predictions increasingly costly. We introduce VFlash, a method for fast, accurate visual planning through Flash Primitive Decoding. VFlash exploits the known structure of visual primitives to predict their coordinates jointly, using a lightweight drafter that generates an entire primitive in one forward pass. The layout adapts to the primitive's type, dimensionality, and length, supporting compact boxes and long, variable-length trajectories. Crucially, spatial accuracy need not require reproducing an autoregressive model's exact token sequence. VFlash therefore trains on annotated primitives and directly accepts structurally valid predictions without target-token agreement. Across the evaluated 2D and 3D robot trajectory tasks, VFlash consistently reduces prediction errors relative to autoregressive decoding. Preliminary single-rollout measurements indicate approximately 25x faster generation for 3D point motion forecasting. A world action model conditioned on VFlash plans achieves 70.2% average success across four MolmoSpaces task suites, compared with 62.6% for the baseline.