SimVLA trains robots in simulation for real-world tasks
SimVLA: Zero-Shot Sim-to-Real VLA Learning for Mobile Manipulation
Robotics
Summary
Collecting real-world data to teach robots is hard and expensive. The authors created SimVLA, a system that trains robots entirely using simulated environments to perform tasks like pouring and cleaning. They built large virtual datasets that give robots both action examples and detailed visual and language guidance. This allows robots to do new tasks in real homes without extra training, sometimes beating robots trained with real data. The work shows simulation can help teach mobile robots complex tasks without needing physical trial and error.
What this means in practice
- •For robotics engineers: Train mobile manipulation robots entirely in simulation to perform complex tasks directly in real-world environments without additional real data collection.
- •For home automation developers: Develop home service robots that can perform restocking, pouring, and cleaning tasks by using robot policies trained only on simulated data.
Authors
Kyoungin Baik, Youngwoon Lee
Abstract
Large-scale, diverse datasets have driven the success of LLMs and VLMs. But VLAs for robotics remain limited by the cost and complexity of real-world data collection. While simulation offers a scalable alternative, its potential for sim-to-real VLA learning in mobile manipulation remains largely underexplored. We introduce SimVLA, an end-to-end framework that trains VLAs entirely on synthetic simulation data without teleoperation for mobile manipulation. SimVLA is first pre-trained on two complementary simulation-derived datasets: SimAction, a large-scale robot action dataset spanning 35 diverse mobile manipulation tasks, generated by composing atomic skills, and SimVQA, which leverages privileged simulator state to provide spatial, geometric, and subtask-level visual-language supervision. We further post-train SimVLA on a mixture of SimAction and SimDeploy, a dataset collected from policy rollouts across diverse simulated environments. We evaluate SimVLA on tasks including restocking, pouring, and cleaning, and show zero-shot transfer to real-world mobile manipulation, including real home environments. SimVLA outperforms policies trained on 50 in-domain real-world demonstrations, suggesting that simulation can enable scalable sim-to-real mobile manipulation. We further demonstrate the value of multiple complementary forms of supervision for effectively leveraging simulation in VLA training.