Vision language speeds up robot training eight times with fewer calls
Scaling Vision-Language Reward Learning for Robot Manipulation in Parallel Simulation
Robotics
Summary
Teaching robots how to do tasks by using vision and language models can be slow and costly because it needs lots of step-by-step feedback. The authors created RAPID, a system that trains robots much faster by running many simulations at once and smartly choosing which images to label. This new method reduces training time from over 9 hours to just about 1 hour and cuts the number of feedback requests dramatically while improving task success rates. They showed these improvements on robot arms performing manipulation tasks in a simulated environment.
What this means in practice
- •For robotics engineers: Train robot arms faster and more efficiently on manipulation tasks by running many simulations in parallel and using smarter feedback from vision-language models.
- •For cloud ai service providers: Reduce latency and cost of automated preference labeling in vision-language models by using single-request prompting methods for label collection.
Authors
Lobna Joualy, Eric Demeester, Nikolaos Tsiogkas
Abstract
Vision-language models (VLMs) can replace human annotators in preference-based reward learning, but sequential API requests and single-environment data collection make training slow and costly. We present RAPID (Reward learning with Adaptive Parallel Image Diversity), a system that couples GPU-parallel rollout with data-aware policy updates, single-request preference labeling, automatic reward stabilization, and representative image sampling. We evaluate these components on five Franka Panda manipulation tasks in IsaacLab. Parallel rollout and adaptive updates provide the first substantial reduction in training time: under matched two-stage prompting, mean runtime falls from 9.18 to 3.13 hours. With all RAPID components enabled, training completes in 1.15 hours using 896 rather than 19,840 API calls per run, and aggregate final success rises from 86.3\% to 98.7\%. This represents an 8.0$\times$ end-to-end speedup and a 95.5\% reduction in API usage. An offline evaluation with Gemma~3 12B and GPT-4.1 mini demonstrates that single-request prompting reduces labeling latency and cost across both models. Code is available at: https://github.com/rapid-vlm/rapid-vlm-rl.