Teacher-in-the-loop method improves robot learning from foundation models
TILDE: Teacher-in-the-Loop Distillation from Embodied Foundation Models
RoboticsMachine Learning
Summary
It is hard to train robots to perform tasks well without lots of expensive data and computation. The authors describe a method called TILDE that helps simpler robot policies learn from powerful pretrained robot models by watching a teacher perform corrections when the student makes mistakes. They also created AutoDAgger, a system that uses a robot’s own reward signals to automatically decide when the teacher should step in without human supervision. This approach improves robot task performance significantly compared to basic imitation learning and speeds up the learning process.
What this means in practice
- •For robotics engineers: Improve robot control policies by transferring skills from pretrained foundation models without requiring extensive retraining data.
- •For automation system integrators: Deploy safer and more robust robots capable of recovering from errors autonomously using teacher-in-the-loop correction methods.
Authors
Yuxiang Lu, Yantong Liu, Jinghua Hou, Xueli Sun, Ruihua Han, Zhe Liu, Hengshuang Zhao
Abstract
Large-scale action pretraining equips Embodied Foundation Models (EFMs) with strong manipulation capabilities, with particular strengths in generalization and closed-loop robustness, but reproducing such pretraining requires substantial robot data and computation. To transfer these capabilities from pretrained EFMs to policies without action pretraining, we introduce TILDE (Teacher-In-the-Loop Distillation from EFMs), which synergizes on-policy distillation at student-induced states with learning from teacher-executed recovery trajectories. We further propose AutoDAgger, an autonomous data aggregation pipeline in which a robotic reward model is utilized to monitor task progress and trigger teacher intervention, thereby collecting recovery experience without human monitoring or manual control. AutoDAgger accelerates recovery-trajectory collection by more than $2\times$ compared with the human-in-the-loop method. By unifying learning from demonstrations, on-policy teacher supervision, and teacher's corrective behavior, TILDE improves task performance by up to $4.1\times$ over behavior cloning across three real-world tasks and supports effective cross-architecture distillation.