Human speeds improve robot task execution success and time
EgoSpeedUp: Transferring Human Manipulation Tempo to Robot Policies
RoboticsComputer Vision and Pattern Recognition
Summary
Robot hands that learn by watching humans usually copy the slow speed of the robot demos they see. The authors found a way to use videos of humans doing the same tasks to teach the robot how fast each step should go. By matching parts of the human and robot actions and speeding up the slow robot demos, the robot gets better at finishing tasks faster and more reliably. They tested this on real robots and saw success rates jump 25% higher while cutting time by over a third.
What this means in practice
- •For industrial robot programmers: Increase manufacturing robot efficiency by speeding up task execution using human demonstration tempos.
- •For robotic assembly line engineers: Optimize assembly robot operation times by retiming robot behavior with human manipulation speed insights.
Authors
Hanbit Oh, Yukiyasu Domae, Takuma Yagi
Abstract
Robot manipulation policies trained through imitation learning inherit not only the demonstrated behavior but also the conservative execution tempo of robot demonstrations. Existing acceleration approaches can execute faster than the original demonstrations, but determine the appropriate acceleration primarily from robot-side information or a predefined set of tempo factors, leaving open how to obtain a task-appropriate reference for how fast each manipulation phase should progress. We introduce EgoSpeedUp, a framework that uses human manipulation as temporal supervision for robot imitation learning. Our key insight is that human demonstrations naturally reveal task-appropriate, phase-wise manipulation tempo. Given slow robot demonstrations and human demonstrations of the same task, EgoSpeedUp aligns corresponding manipulation phases, estimates their relative execution tempos from multiple human demonstrations, and transfers the resulting phase-wise tempo by retiming the robot demonstrations. The retimed demonstrations are then used for standard behavior cloning, allowing the robot to retain its executable manipulation behavior while learning to perform it at a human-informed tempo. Across two real-world manipulation tasks, EgoSpeedUp improves the task success rate by an average of 25 percentage points (pp) while reducing successful execution time by 36.5%. These results demonstrate that human manipulation tempo provides an effective temporal reference for learning faster and more reliable robot policies.