3D human pose estimation models compared for fitness exercise tracking
Comprehensive Evaluation of Monocular and Multi-view 3D Human Pose Estimation Models for Fitness Exercise Analysis
Computer Vision and Pattern Recognition
Summary
Estimating how people move in three dimensions is important for sports and rehab but tricky when using videos from just one or multiple cameras. The authors studied different advanced approaches to see which ones work best for analyzing fitness exercises by testing them on a new dataset with multiple camera views and exact ground-truth positions. They found that the quality of initial 2D pose detection, methods that model motion over time, and post-processing to smooth results all matter a lot. Their work helps show the strengths and limits of these systems in real workout settings.
What this means in practice
- •For fitness app developers: Incorporate the tested 3D pose estimation techniques and filters to improve exercise form analysis and feedback from video input.$Commercial implications: Enables development of commercial fitness coaching apps that provide accurate 3D motion feedback from single or multiple cameras.
- •For physical therapy clinics: Use insights on pose estimation accuracy and temporal smoothing to enhance motion tracking for rehabilitation exercises from video recordings.
Authors
Néstor Ortega, Agustín Salgado, Javier Sánchez
Abstract
Three-dimensional human pose estimation has become a fundamental component of computer vision systems for sports analytics, rehabilitation, and motion understanding. Although recent deep learning approaches have achieved remarkable accuracy on standard benchmarks, their performance under realistic sports scenarios remains insufficiently studied. Furthermore, the trade-offs between recent monocular and multi-view approaches, as well as the benefits of post-processing techniques for improving temporal consistency, have not been systematically evaluated on fitness-oriented datasets. This paper evaluates recent state-of-the-art 3D human pose estimation methods on Fit3D, a dataset of synchronized multi-view recordings of a wide variety of fitness exercises with accurate ground-truth annotations. We study two recent monocular lifting approaches, TCPFormer and PoseMamba-L, and a multi-view model, MVGFormer, under a unified evaluation protocol. To investigate the influence of the input data, monocular methods are evaluated using ground-truth keypoints and detections generated by recent 2D pose estimators, RTMPose and YOLO-Pose. Additionally, we analyze the effectiveness of three temporal filters for reducing prediction jitter while preserving motion dynamics. Experimental results provide a detailed comparison of accuracy, temporal consistency, and robustness to imperfect 2D detections across the evaluated methods. Our analysis highlights the impact of detector quality on lifting-based approaches, compares transformer- and state-space-based temporal architectures, and discusses the practical trade-offs between monocular and multi-view systems for fitness motion analysis. Our results show that 2D detection quality, temporal modeling, and post-processing must be considered jointly in pose estimation pipelines for exercise analysis.