InterPet4D: A Multimodal 4D Human-Pet Interaction Dataset for Pet Motion Generation
2026-07-11 • Computer Vision and Pattern Recognition
Computer Vision and Pattern Recognition
AI summaryⓘ
The authors created InterPet4D, a large dataset of videos and data showing natural interactions between humans and dogs doing obedience tasks. They used multiple cameras and sensors to capture detailed body movements, shapes, and sounds from both the humans and dogs. Using this dataset, the authors developed a new model called InterPetMoGen that can generate realistic human-dog interaction motions, performing better than previous methods. This work helps improve understanding and creation of natural behaviors between people and pets.
datasethuman-pet interactionmulti-view capture3D keypointsmotion generationFID scoremachine learningcomputer visiondogsvideo annotation
Authors
Yichen Peng, Jyun-Ting Song, Chen-Chieh Liao, Kris Kitani, Hideki Koike, Erwin Wu
Abstract
Human-pet interaction estimation and generation remain underexplored due to the absence of a high-quality large-scale dataset. We present InterPet4D, the first multimodal dataset capturing natural interactions between humans and dogs. Using a synchronized multi-view capture system, we record human-dog obedience tasks and provide annotations for both humans and dogs, including multi-view and egocentric videos, segmentations, 2D and 3D keypoints, meshes, and audio tracks. InterPet4D consists of 6.8 million frames collected from 13 dogs of 11 breeds interacting with 23 human participants. We further introduce the InterPetMoGen framework for human-pet interaction motion generation. Our proposed model achieves an FID score of 11.21 and substantially outperforms the Seq2Seq and DiT baselines, demonstrating the effectiveness of InterPet4D for modeling realistic human-pet interactions.