Guided data collection improves robot learning efficiency and performance
GUILD: Guided Iterative Data Collection for Imitation Learning
RoboticsMachine Learning
Summary
Teaching robots to do tasks by watching humans can be slow and expensive because collecting training data is hard and sometimes random. The authors introduce GUILD, a method that smartly chooses which examples robots should learn from by predicting which new data will help their skills most. They use language-based understanding to pick interesting tasks and check how new examples actually improve robot performance, making the training process faster and better. Tests show that this guided approach outperforms ordinary random data collection and works well on real robots.
What this means in practice
- •For robotics engineers: Optimize robotic teleoperation data gathering by selecting tasks that most improve learned policies, reducing training effort and improving deployment success.
- •For industrial automation teams: Enhance robot training pipelines by focusing data collection on relevant environments and goals that increase operational performance.
Authors
Shivin Dass, Kevin Nguyen, Rohan Yelandur, Junhong Xu, Roberto Martín-Martín
Abstract
Imitation learning policies in robotics are trained on datasets collected through costly real-world teleoperation. Often, the data-collection process is unguided, relying on the teleoperator's intuition, resulting in data that contributes little to overall policy performance. Recent work has shown that curating such unguided datasets can substantially improve policy performance, but these approaches operate by filtering existing datasets after collection and therefore do not prevent the acquisition of uninformative data. In this work, we propose GUILD, a novel Guided Data Collection (GDC) method that actively reasons about which data should be collected to maximize downstream policy performance. We introduce an iterative framework that alternates between 1) task proposal and data collection and 2) data evaluation to close the loop. For task proposal, we leverage semantic priors from vision-language models to identify relevant environments, object configurations, and goals for data collection. However, since semantic relevance does not imply policy improvement, we perform data evaluation. By leveraging predictive data attribution we approximate how the collected trajectories influence policy performance on a target deployment setting. This signal is then used as feedback to guide subsequent data collection, closing the data collection loop. We evaluate our approach in simulation and real robot settings, demonstrating that guided data collection substantially improves sample efficiency and deployment performance compared to passive data collection strategies, multiple baselines and experienced roboticists. More information at https://robin-lab.cs.utexas.edu/guild2026/