Robot motions for interacting with objects learned from human contact data
HOI-Retarget: Contact-Centric Retargeting for Human-Object Interaction
Robotics
Summary
Getting robots to do complicated things by copying humans is hard when they have to handle objects the same way people do. The authors created a method to copy the exact points where a human hand or body touches an object, and then adjust the robot’s movement so it can do the same actions smoothly and naturally. Their approach works even when changing the size of the object or the type of robot, and it can use simple video inputs to gather contact information. They also share their tools and examples to help others build on their work.
What this means in practice
- •For robotics developers: Generate realistic whole-body robot motions that interact with objects based on human contact data for diverse robot platforms and object sizes.
- •For animation studios: Create lifelike robot character animations that realistically manipulate objects using retargeted human interaction motions.
Authors
Jihwan Shin, Adrià López Escoriza, Junzhe He, Matthias Heyrman, Marco Hutter
Abstract
Learning from demonstration (LfD) has enabled humanoid robots to acquire diverse whole-body skills, but extending this paradigm to human-object interaction (HOI) is limited by the availability of robot-compatible interaction references. We present HOI-Retarget, a contact-centric retargeting method that transfers HOI onto a humanoid robot for large-scale motion-data generation. Its windowed trajectory optimization uses every labeled contact as a target in the object frame, balancing body tracking, foot support and smoothness under the robot's kinematic limits. The method can augment a single demonstration across object sizes, absorb contacts reconstructed from monocular video, and extend to several robots manipulating one object. We publicly release the code and the retargeted motion dataset.