Eigendexploreshape improves robot hand exploration with human motion data
EigenDEXplore: Structured Exploration for Dexterous Manipulation with Human Priors
RoboticsArtificial Intelligence
Summary
Controlling robot hands to do complex tasks is hard because many fingers need to move together. The authors found that using patterns from human hand motions is best when those patterns guide how the robot explores movements, rather than directly changing how it moves. They created a method called EigenDEXplore that mixes human-inspired joint movements with random changes, helping the robot learn better. This approach worked well for different robot hands doing grasping and manipulation, even when there was little guidance from rewards or examples.
What this means in practice
- •For robotics engineers: Train robot hands to perform complex grasping and manipulation tasks more efficiently using human-motion-guided exploration.
- •For automation integrators: Improve deployment of dexterous robot hands in manufacturing by enhancing control policies with human-inspired movement patterns.
Authors
Harsh Gupta, Tyler Ga Wei Lum, Changhao Wang, Chuer Pan, C. Karen Liu, Jeannette Bohg, Shuran Song
Abstract
Dexterous manipulation poses a challenging high-dimensional optimization problem, as useful behaviors require coordinated motion across many hand joints. In reinforcement learning (RL) and sampling-based trajectory optimization, exploration commonly relies on independent robot joint perturbations, making coordinated behaviors difficult to discover. Prior work reduces this search space for grasp learning using low-dimensional spaces of coordinated joint motions learned from human hand data, but this restricts the expressivity required for general manipulation. Some combine learned and joint-space actions to restore expressivity, but this increases dimensionality and introduces redundancy. We study these effects across diverse manipulation settings, varying action dimensionality, exploration strategy, and the source of human data. Our experiments suggest that human-motion priors are most effective when used to structure exploration rather than change the action representation. Motivated by this finding, we propose EigenDEXplore, which induces correlated exploration by adding perturbations along human-derived eigenvectors to independent joint-space noise, leaving the action space unchanged. Across multiple dexterous hands, EigenDEXplore consistently outperforms joint-space and learned action-space baselines in grasping, in-hand reorientation, and contact-rich manipulation. These gains span unstructured and reference-guided RL, trajectory optimization, and sim-to-real deployment, and are largest in settings with less reward shaping and curriculum design.