Contact anchored retargeting improves robot dexterity in manipulation tasks
CARD: Contact-Anchored Retargeting for Dexterous Manipulation
Robotics
Summary
Teaching robots to use their hands as skillfully as humans is hard because typical methods limit what the human teacher can do. The authors show a new approach called CARD that keeps track not only of hand movements but also of the way hands touch and press objects. This method combines different types of human data, such as video and motion capture, to better copy human hand forces on robots. CARD helps robots maintain real contact with objects, leading to more successful and faithful task performance, and includes a way to reproduce these forces on a real robot.
What this means in practice
- •For robotics engineers: Improve robot hand control by maintaining real contact forces when retargeting human demonstrations from multiple data types.
- •For robotics hardware developers: Use tactile tracking controllers to reproduce demonstrated contact forces during open-loop robot hand replay.
Authors
Cunxi Dai, Botao He, Francois R Hogan, Wei Pan, Rao Fu, Wen-Han Hsieh, Ruoshi Liu, Jitendra Malik, Guanya Shi, Akash Sharma
Abstract
Humans are dexterous, but traditional data collection paradigms such as teleoperation, kinesthetic teaching, and wearable exoskeletons force humans to act through the robot embodiment, bottlenecking their dexterity. Retargeting can avoid this. However, existing methods purely translate kinematics and ignore the real-world contact dynamics essential for task success. We present CARD: Contact-Anchored Retargeting for Dexterous Manipulation, a simulator and object-free framework that retargets human demonstrations to robot actions by explicitly preserving contact force targets. CARD unifies human data sources including video, motion capture, and tactile gloves via a shared hand--contact representation and generates explicit contact force targets in addition to kinematics. To maintain these contact targets during hardware open-loop replay, CARD also introduces a tactile tracking controller. For video which does not contain an explicit tactile source, we present Vid-HACO, our video-to-contact model that sets a new state of the art. Across our video, glove, motion-capture data and the GRAB dataset, CARD maintains contact on $43$--$78\%$ of contact frames, while kinematic and object-aware retargeting methods maintain contact on $<10\%$ and $22.5\%$ of the frames respectively. Further, integrating our formulation into physics-based retargeting improves strict task success from 12\% to 32\% on GRAB dataset. Finally, we show that our tactile tracker is critical to reproduce demonstrated forces during hardware replay. These results establish that contact preservation via CARD is a scalable interface for transferring human dexterity to robots.