Humanoid robots learn object interaction from human motion data
iGPC: Generative Motion Priors for Object-Aware Humanoid Interaction
Robotics
Summary
Humanoid robots need to move naturally and interact safely with objects in their environment, which is very hard to program. The authors build on a system trained on large human motion data to help robots reach, grasp, and push objects using what they call motion priors. They first train experts with detailed environment information to learn specific object tasks, then teach a robot using only its own sensors how to perform these tasks by copying the experts. This method helps robots handle real-world interactions using a foundation of human-like movement.
What this means in practice
- •For robotics engineers: Create robots that physically interact with objects naturally by transferring learned human movements to perception-driven tasks.
- •For automation system integrators: Deploy humanoid robots capable of stable whole-body manipulation in complex environments without detailed environment modeling.
Authors
Anujith Muraleedharan, Abdul Ahad Butt, Nolan Fey, Yash Prabhu, Anamika J H, Sandor Felber, Maurice Rahme, Ivan Laptev
Abstract
Humanoid robots operating in unstructured environments must combine robust whole-body control with the ability to perceive and physically interact with surrounding objects. While large-scale human motion data provides powerful priors for natural and versatile humanoid control, effectively transferring such priors to perception-driven object interaction remains challenging. To address this bottleneck, we propose a framework that extends the recently proposed Generative Pretrained Controller (GPC) from general human motion to full-body humanoid-environment interaction. First, we adapt GPC into interaction experts conditioned on scene affordance cues and privileged state information. These experts leverage the pretrained human motion prior while learning task-specific contact behaviors, including reaching toward objects, grasping environmental supports for stabilization, and pushing movable objects. Second, we introduce a perception-driven student that retains the pretrained GPC policy and distills interaction skills from the experts using onboard sensory observations. To bridge the gap between privileged expert observations and sensory inputs, we propose two complementary training objectives that enable effective adaptation of the pretrained motion prior during distillation. Notably, our experiments across multiple whole-body interaction tasks demonstrate that large-scale generative human motion priors provide an effective foundation for learning deployable policies for humanoid interactions in contact-rich real-world environments.