MEOM: Multi-View Expected-OKS Maximization for Human Pose Triangulation
2026-08-31 • Computer Vision and Pattern Recognition
Computer Vision and Pattern Recognition
AI summaryⓘ
The authors improve 3D human pose estimation by using entire 2D heatmaps from multiple camera views instead of just picking the single strongest point, which can be unreliable when the person is partly hidden. They introduce a method called MEOM to combine heatmaps across views by finding where they agree most. They also use a way to check how trustworthy each heatmap is with HDR calibration. Their approach works well both when they have 3D labels and when they don’t, showing better or comparable accuracy with less complexity.
3D human pose estimationalgebraic triangulation2D keypointsheatmapsmulti-view fusionMEOMhighest-density-region calibrationHuman3.6M datasetMPJPEend-to-end training
Authors
Ziliang Xiong, Henglin Shi, Per-Erik Forssen
Abstract
Conventional algebraic triangulation solves 3D human pose estimation (HPE) from multi-view 2D keypoints. The typical approach, decoding 2D keypoints from predicted heatmaps, is unreliable as heatmaps can be multimodal under occlusion, and collapsing them into single peaks discards their spatial distribution. We seek to use the entire heatmap to estimate 3D poses more accurately, which requires solving two problems: how to robustly fuse heatmaps across views, and how to assess the reliability of heatmaps. For the former, we introduce a novel objective, Multi-viewExpected-OKS Maximization (MEOM), that locates a 3D joint where the views agree in probability mass. For the latter, we adopt highest-density-region (HDR) calibration as a diagnostic of that mass, independently of distance-based metrics. The proposed framework covers two settings, with and without 3D supervision. Without 3D supervision, we optimize 3D poses from pretrained heatmap predictors by maximizing MEOM, achieving comparable performance with state-of-the-art methods that rely on larger backbones, temporal fusion, and simulated 3D data. On ambiguous Human3.6M (H36MA) and occluded CMU Panoptic frames, the advantage is substantial. When 3D labels are available, we train the model end-to-end with a combined MEOM and MSE loss, achieving 19.11 mm absolute MPJPE on Human3.6M outperforming the state-of-the-art volumetric approach on absolute MPJPE at half the inference cost.