Sculpt-vla improves robot task success by staged action learning
SCULPT-VLA: Learning Structured Control through Staged Action Grounding
Robotics
Summary
Robots that understand tasks from vision and language often struggle to decide what to do next. The authors created SCULPT-VLA, which teaches a robot in stages: first by learning a structured understanding of the task, then using that understanding to decide actions, and finally refining actions with direct sensory input. This step-by-step approach leads to better performance in robot tasks compared to methods that learn from vision and language all at once. The method also works without needing extra supervision when deployed.
What this means in practice
- •For robotics engineers: Implement staged action grounding to improve robot manipulation task success rates under diverse visual and language inputs.
- •For automation system developers: Incorporate structured control policies to enable robots to better adapt to dynamic environments without relying on continuous teacher guidance.
Authors
Wenbo Li, Yiteng Chen, Wei Zhang, Wenhao Li, Jun Yang, Qingyao Wu
Abstract
Vision-language-action (VLA) policies increasingly incorporate structured intermediate supervision beyond action labels. Yet specifying what an intermediate representation should encode leaves open how action prediction learns to depend on it. We introduce \textbf{SCULPT-VLA}, a policy that learns structured control through staged action grounding. Its action-conditioning state comprises complementary factors for task progression, scene dynamics, and spatial grounding. Training first forms these factors with teacher scaffolds, then grounds coarse action prediction through their composition as scaffold inputs are withdrawn. Direct perceptual access is subsequently restored for continuous refinement, combining the learned state with perceptual detail. The curriculum separates learning to condition actions on structure from refining continuous control. Deployment requires neither teachers nor discrete-action autoregression. SCULPT-VLA achieves higher average success than shared-backbone baselines on LIBERO, SimplerEnv-WidowX, and RoboTwin 2.0 Full. On SimplerEnv-WidowX, final success is 83.5\%, versus 71.3\% when Stage-II action learning directly accesses vision and language. Across four physical robot tasks, average success under the tested distribution shifts reaches 58.1\%, compared with 45.6\% for $π_{0.5}$. Training ablations and factor-wise interventions support the staged design and show that the learned state continues to contribute to control after direct perceptual access is restored.