Adaptive robot intervention reduces human help in real tasks
UniIntervene++: An Adaptive Intervention Agent for Efficient Real-World Reinforcement Learning
Machine LearningRobotics
Summary
Robots learning to do tasks on their own often need human help, but the amount of help should change as they get better. The authors introduce UniIntervene++, an agent that decides when to assist the robot, how to assist, and when to give control back, adjusting to the robot's improving skills. It tests the robot’s ability by letting it try tasks unaided sometimes and learns from assisted experiences to improve further. This approach led to better success in real-world tasks while dramatically cutting down the need for human intervention.
What this means in practice
- •For robotics engineers: Improve deployment of robot learning systems by dynamically adjusting human assistance based on robot competence in industrial manipulation tasks.
- •For automation integrators: Integrate adaptive intervention schemes to reduce maintenance and human oversight costs when deploying physical robots in uncertain environments.
Authors
Yudong Lin, Haoyuan Deng, Zhuoxuan Yuan, Zaijia Yang, Yuanjiang Xue, Ziwei Wang
Abstract
Online reinforcement learning (RL) enables robot policies to improve through physical interaction, but the assistance they require changes as their competence evolves. Existing intervention strategies based on offline estimates or fixed decision rules can therefore become mismatched to the current policy. To address this, we propose UniIntervene++, an adaptive intervention agent that learns to allocate control between autonomous execution and heterogeneous assisted behaviors during online RL. Specifically, UniIntervene++ first formulates the evolving RL policy, trajectory correction, and a task-structured CodePolicy as Options in a unified semi-Markov decision process and learns their relative values online. Building on this, competence-adaptive intervention periodically probes the RL policy through unassisted execution, keeping control allocation responsive to its evolving capability. Finally, coupled experience learning allows assisted behaviors to improve the RL policy, whose evolving outcomes in turn reshape future intervention decisions. In this way, UniIntervene++ jointly determines when to intervene, how to intervene, and when to return control as the RL policy improves. Across five real-world manipulation tasks, UniIntervene++ achieves an average success rate of 89.67%, outperforming all baselines by at least 6 percentage points, while reducing human intervention to 0.77%, a relative reduction of at least 94.6% from the best baseline. Code is available in our \href{https://github.com/dannyyudong/An-Adaptive-Intervention-Agent-for-Efficient-Real-World-Reinforcement-Learning}{GitHub repository}.