Vision language action models reason with multiple plan possibilities
Reasoning as a Distribution: Generative Plan Latents for Vision-Language-Action Models
Robotics
Summary
Robots and AI systems that see, understand language, and act often have to decide what to do next. Usually, they pick just one plan, hiding other good choices. The authors introduced a new way to keep many possible plans open as a small set of options, letting the system pick, adjust, or learn from different choices later. This helps the AI follow instructions better and improve over time without retraining.
What this means in practice
- •For robotics engineers: Improve robot planning by sampling and selecting from multiple action plans to better follow user instructions and adapt from real-world outcomes.
- •For interactive ai developers: Enable AI agents to steer and revise their planned actions in interactive environments without retraining the model.
Authors
Matteo Cercola, Katie Z Luo, Milan Ganai, John Alora, Simone Formentin, Marco Pavone
Abstract
Vision-Language-Action (VLA) policies increasingly reason before acting, but existing reasoning models compress this intermediate decision into a single representation. When several plans are valid, this commitment hides alternatives that could otherwise be selected or revised. We introduce the Generative Plan Latent (GPL), which represents the reasoning state as a low-dimensional conditional distribution over plans, such as which object, grasp, or route to use. This makes the reasoning state addressable: without retraining the VLA, plans can be sampled (coverage), selected on request (steering), and reweighted by deployment outcomes (outcome adaptation). We test all three on SimplerEnv manipulation with $π_0$, and extend the analysis to RoboCasa, LIBERO, and a maze domain. Steering the plan raises the fraction of executions that follow the request from 0.48 to 0.94, while the same guidance on the native action distribution shows no reliable gain. Finally, we show that an explicit plan is most useful when it carries decision information not already determined by the observation.