FlashDexRetarget speeds up robot hand motion transfer with less training
FlashDexRetarget: Accelerating Dexterous Manipulation Data Generation through Multi-Motion Retargeting
Robotics
Summary
Making robots copy human hand movements is hard because robot hands are different and the motions must be physically possible. The authors designed FlashDexRetarget, a method that uses smart learning tricks to transfer hand-object interactions much faster and more successfully than older techniques. It tracks how objects move and how hands relate to objects, and uses separate models for each hand to speed up learning. Tests show it works well across many motions and different robot hand models with far less computing time.
What this means in practice
- •For robotics development teams: Generate large datasets of physically plausible robot hand movements quickly by adapting human demonstrations to robot-specific hand models.
- •For robotics simulation engineers: Produce high-quality motion retargeting in simulated hand manipulation tasks to test robot control algorithms with varied and realistic motions.
- •For animation studios: Create plausible robotic hand animations from recorded human hand movements efficiently for visual effects and character animations.$Commercial implications: Enables sale of realistic robot hand animation tools for visual media by adapting live hand motion captures directly to robot models.
Authors
Kyungmin Lee, Sibeen Kim, Dongyoon Hwang, Yoonsang Oh, Donghu Kim, Youngdo Lee, I Made Aswin Nahrendra, Jaegul Choo, Hojoon Lee
Abstract
Human hand-object demonstrations offer a reusable source of dexterous robot manipulation data, but transferring them across embodiments requires physically feasible retargeting. Existing physics-based approaches face limitations in retargeting success, motion-specific training efficiency, or both. To address these limitations, we introduce FlashDexRetarget, an RL-based framework for high-success, efficient dexterous motion retargeting. To make the demonstrated interaction easier to learn, we combine object point-cloud observations, hand-object distance features, and future trajectory encodings with complementary rewards that supervise object motion and reference hand-object relationships. To further accelerate learning, we employ separate left- and right-hand actor critic networks and adapt the off-policy algorithm, FlashSAC to dexterous motion tracking. On a benchmark of 50 motions spanning single-object and two-object interactions, FlashDexRetarget achieves a 90% success rate, approximately 2.5x that of the evaluated sampling-based baselines, while requiring up to 100x less training compute than the evaluated RL-based baselines. Evaluations on both XHand and Sharpa Wave Hand show consistent gains, and component-wise ablations examine the contributions of our design choices. Beyond the 50-motion benchmark, experiments with 200, 500, and 1,000 motions demonstrate that our method remains stable at larger scales and produces successful retargeted motions more efficiently as the training set grows. Qualitative replay results using real-world-captured demonstrations further illustrate the applicability of our framework to recorded human manipulation. Videos and code are available at https://davian-robotics.github.io/FlashDexRetarget/