Grasping robot learns new objects during use with over 90 percent success
Continual Learning for 6-DoF Grasp Synthesis via Experience and Demonstrations
RoboticsMachine Learning
Summary
Grasping robots often struggle when they meet new objects they haven’t seen before. The authors created a system that helps a robot learn to grab objects better over time by remembering what worked and what didn’t in past attempts, plus using optional human examples. This system doesn’t retrain the whole robot but updates its memory efficiently to adapt. Tests show it quickly improves grasp success on new items, reaching more than 90% success after just 50 tries.
What this means in practice
- •For robotics engineers: Create grasping robots that improve performance on new objects during deployment without full retraining.
- •For manufacturing automation teams: Deploy robotic arms in variable cluttered environments where objects are frequently changed or unknown ahead of time.
- •For warehouse equipment designers: Design picking robots that adapt from operator demonstrations to handle new product types efficiently.$Commercial implications: Enables commercial robot picking systems that quickly learn new items from few demonstrations, improving service in dynamic warehouses.
Authors
Giulio Schiavi, Andrei Cramariuc, Michael Pantic, Roland Siegwart
Abstract
Most current grasp synthesis systems are trained offline and remain fixed during deployment. While this works well when deployment conditions resemble the training data, performance can degrade when robots encounter conditions they have not seen before, such as unfamiliar objects. In this work, we present a continual-learning framework for single-view 6-DoF grasp synthesis for a parallel-jaw gripper in cluttered scenes. Rather than finetuning a large parametric model, our method adapts through memory in a learned embedding space: grasp outcomes update future grasp scores, while optional user demonstrations are recalled and transferred to new scenes as additional candidate grasps. We evaluate our method in simulation and in extensive real-world experiments comprising over 1500 grasp trials. We show that our method matches the performance of existing 6-DoF grasping baselines even before adaptation, improves online on unseen objects from categories absent or underrepresented during training, and supports long-horizon continual learning with limited forgetting. In real-world experiments, our method reaches over 90\% success rates on several challenging object categories after only 50 online grasp attempts. Videos and code at https://giuschio.github.io/cl_grasping/.