Robot learns efficiently from targeted demonstration requests
DISEIL: Demonstration Distillation for Sample-Efficient Imitation Learning
RoboticsArtificial IntelligenceMachine Learning
Summary
Robots can learn new tasks faster if they know exactly what parts they struggle with and ask for help on those parts. The authors developed a system that watches where a robot first makes mistakes during practice and groups similar errors. Then, using language and vision models, it generates specific requests for demonstrations to fix those errors, avoiding wasted expert time. This method improved learning success in various simulated tasks, especially when only a few demonstrations were allowed. The work focuses on a single practice round in simulation with mostly scripted experts.
What this means in practice
- •For robotics engineers: Improve robot learning speed by automatically requesting targeted demonstrations for specific failure points during training.
- •For industrial automation teams: Reduce expert intervention by focusing demonstration efforts only where robots commonly fail, improving deployment efficiency.
Tested on simulated data.
Authors
Suyog Khanal, Arun Kumar A, Santu Rana
Abstract
A robot that can be taught a new task from a handful of demonstrations has to work out for itself what it still cannot do, and then ask for exactly that. Interactive imitation learning takes a step in that direction by letting a policy practice on its own and calling an expert when it goes wrong. Existing methods decide when to interrupt the learner. A further 2 decisions are left to whichever episode happened to trigger the interruption: which failure to correct, and where the demonstration should start. This paper is a first attempt at making both of them deliberately. DISEIL (Demonstration dIstillation for Sample-Efficient Imitation Learning) marks each failed episode at the step where the policy first becomes unreliable, represents that moment with a geometric descriptor, and groups the failures into recurring failure modes. A vision-language model and a language model read the selected mode and write a request for the next demonstration, and a store of task constraints checks that the request can be carried out before any expert time is spent. No model produces a robot action. Across 5 simulated tasks under state and image observations, changing only what the expert is asked for gives the highest mean held-out success rate in all 10 settings, with a tie in 1, and the margin is widest at the smallest budget we tested. The scope is narrow: a single round of practice at a time, in simulation, with experts that are mostly scripted. The longer-term aim is a learner that also tracks what its demonstration set already covers, and that asks a human teacher for the missing behavior in proportion to the effort each request costs them.