Prosthesis aware 3D human pose estimation improves motion capture
Prosthesis-Aware 3D Human Pose Estimation: A Dataset and Benchmark for RSP Users
Computer Vision and Pattern Recognition
Summary
Capturing 3D body movement from video helps with things like rehab and sports training but is hard for people with prosthetic limbs. Existing methods don’t work well because they don’t understand prosthesis shapes or movement. The authors created a new dataset of people using running-specific prostheses (RSPs) to help machines learn these special cases. They tested current methods and introduced a combined approach to better estimate the body and prosthetic pose together.
What this means in practice
- •For rehabilitation specialists: Use improved 3D pose estimates for better assessment of prosthesis users during therapy and exercise.
- •For sports performance coaches: Analyze athletes with running-specific prostheses using more accurate 3D motion data to guide training.
Authors
Yilin Wen, Kechuan Dong, Fumiya Suginaka, Ken Endo, Yusuke Sugano
Abstract
Recovering 3D human body motion from video is important for applications such as rehabilitation assessment and sports performance evaluation. For prosthesis users, this requires capturing both natural body joints and the geometry of the prosthetic device, a challenge that existing methods are not designed to address. Model-based estimators rely on body models trained on non-amputee individuals and cannot represent prosthesis geometry, while model-free methods lack body kinematic priors and are unreliable under occlusion. This challenge is particularly prominent for users of running-specific prostheses (RSPs), where the RSP has a complex curved geometry and moves dynamically during exercise. To fill this gap, we collect RSP3D, the first 3D dataset of RSP users, covering essential daily-life and exercise actions from participants with varied amputation conditions, using a multi-camera marker-based motion capture setup. We formally define the task of prosthesis-aware 3D pose estimation, evaluate representative methods in a zero-shot setting, and confirm their individual limitations. We further propose a hybrid baseline combining model-based body joint estimation with model-free RSP shape recovery, establishing a starting point for future research.