Humanoid robots mimic human interactions from wild videos
Mimetic Embodiment Transfer of Human-Humanoid Interaction from In-the-Wild Videos
Robotics
Summary
Getting robots to interact naturally with people means not only copying their moves but also understanding how their bodies relate during actions like hugging or dancing. The authors created HumiTransfer, which turns regular videos of humans interacting into robot motions that look real and keep the important body contacts and positioning. They tested this on two different robots and made a huge dataset of these interactions. Their method works better than others and the robots can physically perform these motions in the real world.
What this means in practice
- •For robotics engineers: Generate physically valid humanoid robot interactions from real-world human videos for improved robot behavior in social scenarios.
- •For animation studios: Create realistic humanoid character animations by transferring natural human interactions captured in common videos onto diverse digital models.
Authors
Nghia Le, Khang Nguyen, Ba Long Tran, Thi Quynh Nhu Pham, Binh Nguyen, Tuong Do, Erman Tjiputra, Hoan Nguyen, Tung D. Ta, Baoru Huang, Hai-Nguyen, Nguyen, Anh Nguyen
Abstract
Enabling a humanoid to interact naturally with human partners requires preserving not only individual motions but also the spatial, semantic, and contact relationships underlying the interaction (i.e., hugging, dancing, hand-shaking, etc.). Furthermore, a practical solution should generalize across various humanoid embodiments with distinct morphologies and kinematic constraints. To tackle this challenge, we propose HumiTransfer, a mimetic embodiment transfer framework that converts in-the-wild human-human interaction videos into physically plausible human-humanoid interactions. HumiTransfer first reconstructs 3D human motion with consistent foot-ground contact, then transfers one participant to a target humanoid embodiment via interaction modeling, using weighted joint-relative directions and anatomical offsets to prioritize interaction-critical inter-body relationships, while post-retargeting optimization enforces balance, contact, and collision avoidance. Subsequently, we apply HumiTransfer to 1,200 in-the-wild videos, producing approximately 500,000 frames of diverse interactions for the 26-DoF Unitree R1 Edu and the 23-DoF HighTorque Mini-hi humanoid robot, resulting in our HumiMimic dataset. Across both robots, HumiTransfer achieves superior motion fidelity, preserves inter-body relationships, and maintains contact accuracy compared with existing baselines. Moreover, we demonstrate that HumiMimic supports motion generation methods for synthesizing interactive humanoid motions following a human partner. Finally, our real robot deployments confirm the physical executability of the transferred motions in real-world settings.