Radar improves 3D human pose tracking by combining motions and observations
Radar-Based 3D Human Pose Estimation via Motion-Aware Pose Initialization and Observation-Guided Refinement
Computer Vision and Pattern RecognitionArtificial Intelligence
Summary
Estimating a person’s 3D pose inside a room using radar is hard because some body parts may be hidden or confuse the radar signals. The authors created a two-step method where first they guess a person’s movement by looking at radar data from time and using knowledge about the human skeleton. Then, they refine this guess by checking the radar measurements more closely. They tested their method on new and existing radar datasets and showed it predicts body poses more accurately than previous methods.
What this means in practice
- •For smart home developers: Use radar-based pose estimation to monitor residents’ activities indoors without cameras, even in low light or privacy-sensitive areas.$Commercial implications: Enables development of privacy-respecting smart home systems for activity monitoring and fall detection using mmWave radar.
- •For roboticists: Enhance indoor robots’ understanding of human poses from radar data where vision sensors may fail due to poor lighting or occlusions.
Authors
Woo-Jin Jung, Chan-Bin Lim, Seung-Hyun Kong
Abstract
3D Human Pose Estimation (3D HPE) is used in indoor environments, such as smart homes, to analyze human movement and monitor human states. Millimeter-wave (mmWave) radar is suitable for this task because it is robust to lighting variations and does not capture images of human appearance. However, radar observations often provide insufficient information about certain body parts or contain overlapping reflections from multiple body parts, limiting accurate 3D joint localization. To address these limitations, we propose the Initialization-to-Refinement Pose Framework (I2R-Pose), comprising two stages: Motion-Aware Pose Initialization and Observation-Guided Refinement. In the first stage, joint-wise features are learned by incorporating temporal relationships across consecutive radar observations and human skeletal structure to compensate for insufficient joint information and estimate an initial 3D pose sequence. In the second stage, separately extracted radar observation features predict joint-wise positional corrections that refine the initial pose and generate the final pose sequence. The framework supports individual and fused inputs of time-Doppler/time-range (TD/TR) maps and radar point clouds (PCs). To evaluate performance across indoor activities, we construct the Radar Motion and Spatial Information for 3D Human Pose Estimation (RaMoS-Pose) dataset, providing synchronized TD/TR maps, radar PCs, and 3D joint ground truth (GT). Evaluations on RaMoS-Pose and the publicly available MVDoppler-Pose dataset show that I2R-Pose reduces mean per-joint position error (MPJPE) by 39.50% on MVDoppler-Pose and by 15.13% on RaMoS-Pose with fused inputs, compared with the best-performing existing methods evaluated on each dataset.