Robot learns new tasks faster by asking for targeted help
DISEIL: Demonstration Distillation for Sample-Efficient Imitation Learning
RoboticsArtificial IntelligenceMachine Learning
Summary
Robots need to learn from a few examples how to do new tasks, but this can be challenging when they don't know exactly what they need to improve. The paper presents a method called DISEIL that helps robots identify the precise moment they fail and group similar failures together. It then uses language models to generate specific requests for demonstrations that address these failures, making the learning process more efficient. This approach was tested in simulations and improved success rates, especially when the number of demonstrations was limited. The system aims to minimize effort from human teachers by only asking for the most helpful demonstrations.
imitation learningrobot learningdemonstration distillationinteractive learningfailure modesvision-language modellanguage modelsample efficiencytask constraintssimulation
Authors
Suyog Khanal, Arun Kumar A, Santu Rana
Abstract
A robot that can be taught a new task from a handful of demonstrations has to work out for itself what it still cannot do, and then ask for exactly that. Interactive imitation learning takes a step in that direction by letting a policy practice on its own and calling an expert when it goes wrong. Existing methods decide when to interrupt the learner. A further 2 decisions are left to whichever episode happened to trigger the interruption: which failure to correct, and where the demonstration should start. This paper is a first attempt at making both of them deliberately. DISEIL (Demonstration dIstillation for Sample-Efficient Imitation Learning) marks each failed episode at the step where the policy first becomes unreliable, represents that moment with a geometric descriptor, and groups the failures into recurring failure modes. A vision-language model and a language model read the selected mode and write a request for the next demonstration, and a store of task constraints checks that the request can be carried out before any expert time is spent. No model produces a robot action. Across 5 simulated tasks under state and image observations, changing only what the expert is asked for gives the highest mean held-out success rate in all 10 settings, with a tie in 1, and the margin is widest at the smallest budget we tested. The scope is narrow: a single round of practice at a time, in simulation, with experts that are mostly scripted. The longer-term aim is a learner that also tracks what its demonstration set already covers, and that asks a human teacher for the missing behavior in proportion to the effort each request costs them.