Hierarchical learning method enables efficient dexterous robot hand control
HiFun: a Hierarchical Framework for Efficient Functional Dexterous Manipulation Learning
Robotics
Summary
Controlling robot hands with many fingers to do delicate tasks is very hard because it requires precise finger movements and careful coordination with the robot arm. The authors propose a new way to teach robots these skills by breaking the problem into two parts: one part focuses on controlling the fingers precisely, and the other part controls the arm and decides when to use the finger skills. This approach helps robots learn faster in the real world, recover from mistakes, and handle disturbances. They tested this method on different tasks and robot hands, achieving high success rates after only one hour of training.
What this means in practice
- •For robotics engineers: Develop more effective control systems for robot hands that learn complex tasks quickly in real-world environments.
- •For manufacturing automation teams: Deploy robot hands capable of precise, adaptive manipulation for assembly and handling tasks requiring fine dexterity and error recovery.
Authors
Linyi Huang, Guowei Huai, Weibin Liu, Shulong Jiang, Ping Tan, Weixuan Zhang, Hui Zhang, Jie Song
Abstract
While multi-fingered dexterous robot hands have the potential to provide human-level dexterity, using them for functional manipulation remains difficult because task success depends on precise contact, coordinated arm-hand motion, and reactive recovery from execution errors. Precise contact depends on complex contact dynamics that are difficult to model accurately in simulation, motivating real-world learning. However, real-world demonstrations in the full arm-hand control space are costly and noisy because each trajectory must coordinate arm motion with high-DoF multi-finger contact. Small execution deviations can further cause slipped contact or premature force application, making recovery from off-nominal contact states important for robust execution. To deal with these challenges, we propose HiFun, a hierarchical real-world RL framework for functional dexterous manipulation tasks. Our key insight is to decouple the control complexity into a functional hand skill for precise contact execution and an arm-hand coordination policy for arm control and hand-skill activation. This design enables effective real-time human intervention for reactive recovery learning and efficient data collection with Dynamic Movement Primitive (DMP) augmentation. Across six real-world tasks on two dexterous hands, HiFun achieves 98.3% average success within one hour of online training and remains robust to dynamic disturbances. The project page is available at https://hly-123.github.io/HiFun/.