HybridSim: A Physics-Learning Hybrid Digital Twin for mmWave Human Sensing
Computer Vision and Pattern Recognition
Summary
The authors created HybridSim, a new way to simulate mmWave radar signals that come from people moving in a fixed indoor space. Their method splits the radar signal into direct reflections from the person and indirect reflections caused by the room, making the simulation more accurate and faster than traditional approaches. They use computer techniques to model the person's shape and movements and combine this with a smart way to guess the complex signal paths in the room. Tests showed their simulator matches real measurements better and helps improve radar-based human sensing models when generating new training data.
Authors
Weitao Xiong, Tianyu Liu, Peng Li, Kok Chung Chua, Toa Chean Khim, Pu Wang, Hongfei Xue
Abstract
High-fidelity simulation of mmWave radar signals for dynamic human motion is valuable for developing radar-based human sensing models; yet collecting accurately labeled measurements for a specific deployment site remains expensive. We present HybridSim, a physics-learning hybrid simulator that synthesizes mmWave radar signals from dynamic human meshes under a fixed indoor room configuration, explicitly decoupling propagation into two components. To parameterize the human subject, we use a tri-plane representation to extract human features and a Graph Convolutional Network to stabilize optimization and mitigate gradient instability. The direct signal path is modeled via an inverse-rendering formulation with a microfacet BRDF to capture primary surface reflections. In parallel, the indirect path is approximated by combining 3D Gaussian Splatting with a virtual-receiver geometry to fit and reproduce site-specific multipath interference patterns, achieving substantially lower computational cost than explicit full ray tracing. Experiments in a fixed-room setting show improved agreement with a physically based reference and consistent gains on downstream radar-based human sensing tasks when using HybridSim for site-specific data augmentation.