Robotic hand learns to assemble two parts using one hand
Assembling Two Parts in One Hand
Robotics
Summary
Putting two objects together using only one hand is something humans do with ease, but it’s hard for robots. The authors taught a robot hand to hold and fit two objects together without needing help from another arm or tool. They used a kind of trial-and-error learning method, trained the robot in a computer simulation, and then successfully tested it on a real robot hand. The method works even when the robot can’t perfectly see the objects it’s holding, showing it can handle tricky situations like occlusion.
What this means in practice
- •For robotics engineers: Build robotic hands that can assemble parts within one hand, improving autonomy for manufacturing robots handling small components.
- •For robotic system integrators: Deploy vision-based dexterous manipulation policies that work reliably despite occlusions, enabling more flexible manufacturing setups with fewer sensors.
- •For prosthetics developers: Design advanced prosthetic hands capable of coordinating fingers to assemble objects in hand, enhancing the usability and dexterity for users.$Commercial implications: This paper enables prosthetic hands to perform complex assembly tasks without external aids, offering improved daily life functionality.
Authors
Liuao Pei, Tianyue Wu, Hui Zhang, Ping Luo, Jie Song
Abstract
A hallmark of human dexterity is the cooperative use of fingers, where different fingers take on distinct yet coordinated roles to accomplish fine manipu- lation, such as capping a pen with the hand that holds it. We study this finger-level coordination through in-hand assembly: mating two rigid objects within a single dexterous hand, with no second arm and no fixture. We present a reinforcement learning formulation to solve this problem in a unified framework, which is driven by a goal relative pose between the two parts. Finger coordination is shaped by a function-based auxiliary reward and regularized toward a single human reference pose, while domain randomization and a fusion of historical proprioception and object observation confer robustness to occlusion-induced estimation noise. The same recipe solves three different assembly tasks (Bottle, Syringe, and Marker). Trained purely in simulation, the policies transfer zero-shot to hardware with a single camera, demonstrating robustness to state-estimation errors caused by oc- clusion. Our experiments also reveal that in-hand assembly places demands on hand morphology and can serve as a benchmark for modern robotic hand systems. Videos and code are available at https://ltbgbird.github.io/in-hand-assembly-page/.