Multiview Multi-Person Human Mesh Recovery Under Large Scenes with Occlusions
2026-07-27 • Computer Vision and Pattern Recognition
Computer Vision and Pattern Recognition
AI summaryⓘ
The authors created a big new test set called MVMP-HMR to help computers learn how to build 3D models of many people in large spaces with lots of camera views and people blocking each other. They also made a new method that combines information from many cameras into a 3D space and uses predicted body joints to focus on each person separately. Their method includes special rules to better guess body orientations and poses when people overlap or are partly hidden. Tests show that the new dataset is much harder than before and their method works better than others in these tough situations.
Human Mesh Recovery3D Pose EstimationMultiview ReconstructionOcclusionSynthetic Dataset3D Feature VolumeCross-AttentionOrientation Loss3D Joint Density LossMulti-person 3D Reconstruction
Authors
Qi Zhang, Tao Yu, Jiechao He, Antoni B. Chan, Hui Huang
Abstract
Human mesh recovery (HMR) aims to recover 3D human meshes from images. Most existing HMR benchmarks and methods focus on either multi-person reconstruction from a single view or single-person reconstruction from multiple views, where the number of subjects and the scene scale are relatively limited. Such settings are insufficient for real-world applications with large scenes and severe inter-person occlusions. To address this limitation, we introduce a large-scale synthetic benchmark for multiview multi-person HMR, termed MVMP-HMR. The proposed dataset contains 15 complex scenes with up to 50 camera views and 30 interacting persons, featuring large spatial coverage and severe occlusions, which significantly increases the difficulty of human mesh recovery. Based on this benchmark, we further propose a multiview multi-person whole-body human mesh recovery model, referred to as MVMP-HMR model. The model first fuses multiview features into a scene-level 3D feature volume, and then leverages pelvis joints predicted by a 3D pose estimation network to extract person-specific queries from the 3D feature volume. These human queries are cross-attended with the 3D feature volume and integrated to decode each person's 3D mesh. Moreover, we introduce two novel losses--the orientation loss and the 3D joint density loss--to alleviate orientation and pose ambiguities under severe occlusions. Experiments demonstrate that existing state-of-the-art HMR methods struggle on the proposed MVMP-HMR benchmark, while our method consistently outperforms prior SOTAs in large-scale scenes with severe occlusions.