ContactMimic: Humanoid Object Interaction via Contact Control
2026-07-09 • Robotics
Robotics
AI summaryⓘ
The authors found that just tracking points on objects isn't enough for robots to interact properly with things like chairs or boards. They created CONTACTMIMIC, a system that teaches robots to track both these points and when to touch parts of the object. This helps robots make real physical contact and control when they touch or don’t touch during tasks. Their tests in simulations and real life showed that this approach works better than just tracking points and can handle multiple tasks without extra guidance.
Keypoint trackingContact controlRobot manipulationTrajectory augmentationSim-to-real transferBinary contact commandsPhysical interactionReward functionsHuman-object interaction
Authors
Xinyao Li, Xialin He, Runpei Dong, Saurabh Gupta
Abstract
Keypoint tracking alone is insufficient for object interaction tasks such as sitting on a chair, wiping a board, or pushing furniture, where the robot can reach the correct pose without making meaningful physical contact with the object. We present CONTACTMIMIC, a learning framework that tracks explicit partlevel binary contact commands alongside keypoint trajectories. CONTACTMIMIC is made possible through the use of contact-following rewards and a trajectory augmentation scheme aimed at breaking the correlations between keypoint trajectories and contact labels. The resulting policy successfully decouples contact behavior from keypoint geometry, and achieves precise physical contact as well as contact-controllability (produce or suppress contact during deployment as desired). Simulation experiments across 10 diverse human-object interaction motions confirm that CONTACTMIMIC exhibits contact controllability that enables it to complete manipulation tasks without task-specific rewards, while also outperforming keypoint-only trackers on contact-relevant tasks. Ablations confirm the necessity of the proposed trajectory augmentation scheme and sim2real deployment validates contact controllability in the real world across 5 different motions. Video results are available on https://lixinyao11.github.io/contactmimic-page/.