AstroPIE-53 reveals challenges for astronaut pose detection in space
AstroPIE-53: Towards Understanding Failure Modes of Human Pose Estimation in Space
Computer Vision and Pattern Recognition
Summary
It is important to watch astronauts carefully to keep them safe during space missions. However, current computer vision tools that work well on Earth face many problems when used in space. The authors created a special dataset called AstroPIE-53 using publicly available videos of astronauts, helping study when and why these tools fail. They found that issues like unusual body positions, low lighting, and floating movements in microgravity cause these tools to perform poorly. This work helps identify what needs fixing for better astronaut monitoring in future missions.
What this means in practice
- •For space mission developers: Test human pose detection tools in space conditions to improve astronaut monitoring systems.
- •For robotics engineers: Design assistive robots that better handle human detection challenges in low-contrast and unusual orientations.
Tested on one dataset.
Authors
Stephanie Käs, Marten Flaig, Lukas Woyke, Osmanalp Ömeroğlu, Felix Krueckel, Onur Alp Çulha, Bastian Leibe
Abstract
Reliable monitoring of astronauts is essential for safety, situational awareness, and autonomous assistance in space missions. While modern computer vision enables accurate human detection and pose estimation on Earth, its robustness in space environments remains largely unquantified due to limited data availability. We present AstroPIE-53, a pilot dataset of 53 annotated astronaut videos collected from publicly available mission footage. To respect copyright restrictions, we release video identifiers, reconstruction scripts, and dense annotations rather than raw videos. The dataset includes bounding boxes, challenge classes, and human-annotated pose failure tags. We benchmark state-of-the-art person detection and pose estimation models under realistic zero-shot deployment conditions and perform the first systematic failure analysis for astronaut monitoring. Our results reveal substantial performance degradation caused by occlusions, unusual body orientations, low contrast, and microgravity motion patterns.