Bimanual robot learns to open jars with finger reconfiguration
Grasp2Twist: Learning Bimanual Dexterous Jar Opening by Reinforcement Learning
Robotics
Summary
Opening jars with robot hands is tricky because it involves multiple steps and adjusting finger positions. The authors created a robot system that learns to grasp jar lids and twist them open by practicing in simulation and then applying the skills directly in real life. Their method uses the shape and position of the hand and jar to guide when to switch from grabbing to twisting, and a step-by-step learning process to master finger movements needed for twisting. This approach achieved high success on common household jars without extra real-world training.
What this means in practice
- •For industrial automation teams: Program robot arms to open various container lids using learned finger repositioning for sustained twisting motions.
- •For household robot developers: Integrate bimanual dexterous manipulation policies enabling robots to open jars zero-shot without real-world retraining.$Commercial implications: Enables consumer robots to perform a common household task more reliably, improving market readiness for assistive devices.
Authors
Mo Xu, Yunfu Deng, Jianuo Wang, Josiah Hanna, Bilge Mutlu
Abstract
This paper presents Grasp2Twist, a bimanual dexterous manipulation system that learns to grasp and twist open jar lids using reinforcement learning. Learning this task raises three challenges: learning a unified policy for a multi-stage task, sustaining lid twisting, and sim-to-real transfer. To address the first challenge, we introduce a continuous enclosure measure to guide grasp formation and a binary enclosure indicator to guide the grasp-to-twist transition for unified policy learning. We derive both from the geometric relationship between the object center and the convex hull formed by the hand's palm and fingertips. Kinematic constraints limit how far the hand can rotate the lid with fixed contacts, so sustained twisting requires finger contact reconfiguration. We use a three-stage curriculum to facilitate exploration of these contact changes and also improve robustness for sim-to-real transfer. With our approach, the learned policy demonstrates finger gaiting, reconfiguring hand-object contacts to sustain lid rotation. It transfers zero-shot to the physical system and achieves an 88% task success rate across six household containers, including peanut-butter, vitamin, and instant-coffee jars. Ablations further validate the roles of the geometric enclosure in grasp formation and the curriculum in contact-reconfiguration exploration.