Vision language action models improved for robust robot tasks in varied environments
Red-Teaming of Vision-Language-Action Models via Quality Diversity Environment Generation
Robotics
Summary
Robots that understand both images and text to perform tasks can struggle when things change a little, like moving objects or lighting. The authors created a method called Q-DEG that finds tricky environments where the robot might fail and uses those to help the robot learn better. This method helps the robot get better at handling unexpected changes and also transfers improvements from simulation to real-world situations.
What this means in practice
- •For robotics engineers: Create robots more robust to environment changes by generating diverse challenging test settings and improving robot learning from those failures.
- •For automation system integrators: Enhance simulated training workflows to better prepare robots for real deployment with unpredictable physical conditions.
Authors
Shihan Zhao, Henry Chen, Ya-Chuan Hsu, Minjune Hwang, Siddharth Srikanth, Freddie Liang, Akanksha Saran, Aaquib Tabrez, Stefanos Nikolaidis, Daniel Seita
Abstract
A long-standing goal in robotics is to develop generalist robots that can perform tasks across a wide range of settings, flexibly specified with images and text. Vision-Language-Action (VLA) models offer a promising path toward this goal by training on large collections of robot demonstrations. However, while VLAs perform well in-distribution, they often fail under minor test-time shifts, such as changes in object positions, lighting, textures, and camera poses. To address this, we propose Quality Diversity for Diverse Environment Generation (Q-DEG), a method for improving VLA robustness by generating adversarial environments and fine-tuning the VLA on its own successful rollouts. Q-DEG uses Quality Diversity (QD) optimization to find a set of high-failure environments, while promoting diversity using policy embeddings extracted from the VLA. These environments expose vulnerabilities that can be mitigated through self-imitation fine-tuning, thereby improving the VLA's robustness. In simulation, we conduct experiments across three representative VLA models, and our results indicate that Q-DEG improves robustness to out-of-distribution (OOD) adversarial environments compared to the base VLA and VLAs fine-tuned using data collected from two red-teaming baselines. Furthermore, we demonstrate that improving VLA robustness through Q-DEG facilitates sim-to-real transfer. Our anonymous project website is at: https://qd-env.github.io/qdeg/.