Whole body control lets robots hold and move bulky objects easily
Whole-Body Dexterity
RoboticsArtificial IntelligenceMachine Learning
Summary
Typical robots only use their hands to hold things, unlike humans who use their arms and bodies to handle big or awkward stuff. The authors developed a method that helps robots use their whole body to grab and move objects by learning many possible ways to hold them. They trained and combined multiple robot experts in simulation, and then tested the combined policy on a real robot successfully moving a basketball and a box around its body. This method worked much better than previous ones and lets a robot perform complex tasks like carrying and placing a box using its arms and torso.
What this means in practice
- •For industrial robot integrators: Enable robots to use their arms and torso for better handling and repositioning of large or bulky objects in warehouses and factories.
- •For robotics software developers: Develop interactive robot controllers that use whole-body dexterity and teleoperation for complex manipulation tasks.
Authors
Conor Mc Gartoll, An Dang, Pranay Thangeda, Arjun Gupta, Carolina Higuera, Nima Fazeli, Mustafa Mukadam, Manikantan Nambi
Abstract
Current humanoid dexterous manipulation remains largely confined to end-effectors and simple grasps, falling short of humans' ability to engage the entire body by bracing and sliding bulky objects against the arms and torso. We introduce Whole-Body Dexterity (WBD), a general framework that uses the robot's body and appendages as contact surfaces to grasp and manipulate objects. Our sim-to-real pipeline first generates a dataset of feasible whole-body grasps, then trains reinforcement-learning experts on a general any-to-any repositioning objective by sampling initial and goal grasps from the dataset, and distills these experts into a single policy that uses proprioception and object geometry. We show that the distilled policy transfers zero-shot to a real Unitree G1, where it repositions a basketball and a cardboard box to regions around the torso and completes a long-horizon task that stows a box under one arm, carries it from a shelf to a table, and places it. The controller operates interactively, steered by coarse motion references from a human teleoperator. Across real-world whole-body manipulation tasks, the distilled controller completes 90% of trials, compared with 18% for direct retargeting and 4% for a generalist whole-body controller.