Robot learns when to ask for help using progress signals
Reward-DAgger: Robot-Gated Interactive Imitation Learning with General-Purpose Progress-Based Reward Models
RoboticsArtificial Intelligence
Summary
Robots can struggle when trying tasks they've never seen before and might fail without knowing it. The authors created a method called Reward-DAgger that helps robots decide when to ask humans for help by measuring how much progress they are making, using a general reward model. This way, the robot can safely learn from human guidance only when necessary, improving its ability to succeed on many different tasks without complicated setups. The method works well across multiple environments and robot types without needing special tuning for each case.
What this means in practice
- •For robot developers: Automate robot failure detection during varied task execution by integrating a progress-based signal for timely human interventions.
- •For industrial automation engineers: Enhance reliability of robots working in changing environments by using a task-independent method to gate human corrections without retuning.
Authors
Ryan Li, Yigit Korkmaz, Erdem Bıyık
Abstract
Recent advances in robot learning have enabled generalist control policies capable of completing a wide range of tasks. However, their performance degrades when deployed in unseen environments, making it critical to detect failures and teach recovery behaviors. Existing runtime monitoring methods often require task- and policy-specific training or hyperparameter tuning, limiting cross-task deployment and introducing additional overhead during iterative policy updates. We present Reward-DAgger, a robot-gated interactive imitation learning framework that uses dense progress signals from a general-purpose reward model to determine when human intervention is needed. Our approach is agnostic to the underlying policy architecture, requires no access to policy internals, and can be applied across tasks without retuning the gating mechanism. Our results show that Reward-DAgger achieves a better failure-detection accuracy-latency tradeoff than existing runtime monitoring baselines. Across eight simulated and real-world tasks, Reward-DAgger consistently improves the downstream policy's autonomous success rate throughout interactive learning and achieves strong return on human effort, outperforming the baselines in most settings. Importantly, the same gating configuration is used across tasks without task-specific hyperparameter tuning, demonstrating transfer across tasks, environments, and policy architectures. Code and videos are available at https://liralab.usc.edu/reward-dagger.