Exoskeleton shared by human and robot improves dexterous hand learning
SEED-UMI: Sharing the Exoskeleton between human and robot for onE-to-one Dexterous demonstration
Robotics
Summary
Teaching robots to use dexterous hands is hard because it's difficult to collect detailed demonstrations involving touch that robots can copy accurately. The authors created SEED-UMI, where both a human and a robot wear the same exoskeleton glove, sharing joint measurements and wrist camera views. This setup helps the robot learn more directly and effectively from the human movements, improving learning efficiency and success in handling tasks that require careful touch.
What this means in practice
- •For robotics engineers: Collect high-quality demonstration data that directly matches robot hand movements for training dexterous manipulation policies in contact-rich environments.
- •For robotic prosthetics developers: Use shared exoskeleton sensing to improve the control and learning of prosthetic hands by better correlating human and device motions through shared measurement.
Authors
Tengbo Yu, Jiahao Wu, Daohan Li, Bingxu Chen, Hao Liu, Xiaojian Ma, Hangxin Liu
Abstract
Imitation learning for dexterous hands is bottlenecked by the difficulty of collecting contact-rich demonstrations that transfer faithfully to the robot. Prior wearable-exoskeleton systems record only on the human side and retarget via open-loop mappings calibrated in free space, which degrade under contact. We present SEED-UMI, a framework in which both the human and the robot wear the same exoskeleton: joint encoders become a physically shared measurement, and wrist cameras mounted to the exoskeleton observe the same outer mechanism during both human data collection and robot policy rollouts. This turns retargeting into paired cross-embodiment supervision and lets policies train directly on raw exoskeleton-centric wrist images, without segmentation or inpainting. On five contact-rich tasks, SEED-UMI achieves 3.0x greater data collection efficiency than exoskeleton-based teleoperation and a 70.0% average rollout success rate.