Radar improves moving object tracking in challenging surveillance scenes
Segment Any Motion with Radar: Robust Multimodal Moving-Object Segmentation and Tracking
Computer Vision and Pattern RecognitionArtificial Intelligence
Summary
Tracking moving objects in video is hard when lighting is bad, the weather is poor, or objects block each other. The authors introduce a new dataset that combines regular video, thermal images, and radar data to detect and follow moving things more reliably. They build a system called SAM-Radar that uses radar signals along with images to spot and track objects, even when visuals are unclear. Their method keeps objects identified consistently over time, leading to better tracking than previous approaches.
What this means in practice
- •For security surveillance teams: Enhance surveillance systems with radar and thermal fusion to maintain accurate tracking of individuals under poor visibility conditions.
- •For autonomous vehicle engineers: Use radar-augmented perception to improve object detection and tracking despite occlusion or bad lighting.
Authors
Jue Wang, Xuan Wang, Hao Zhou, Ruixiang Zhou, Yixuan Zhou, Tianshuo Yuan, Jieming Ma, Jie Zhang, Fei Luo
Abstract
Moving-object perception must decide which image regions correspond to real motion and keep every instance identified over time. Methods that read motion from appearance, optical flow, or estimated trajectories lose that evidence under poor illumination, adverse weather, reflections, and occlusion. Radar is a natural remedy because it measures radial velocity directly instead of inferring it from photometric correspondence. However, existing benchmarks do not jointly provide radar measurements, dense moving-instance masks, and temporally consistent identities for surveillance. We therefore introduce RGBTR-Motion, a synchronized and calibrated fixed-camera benchmark that pairs RGB, thermal, and radar streams with dense instance masks and temporally consistent identities across diverse surveillance scenes. We also develop SAM-Radar, an RGB, thermal, and radar-based segmentation and tracking framework built on SAM 3. SAM-Radar's radar-aware detector fuses calibrated RGBT features with radar returns that are grounded at their projected image locations, and motion supervision, implemented as foreground classification of those projected returns, teaches the detector to reject clutter without any text prompt. The tracker associates accepted radar returns with individual trajectories and uses them as physical evidence that a visually degraded target remains present. This allows it to bridge short periods of low visibility or occlusion and reconnect a reappearing target to its existing identity instead of starting a new track. SAM-Radar attains 0.7027 IoU and 0.8090 F1-50, and raises MOTA, HOTA, and IDF1 by 0.2977, 0.1603, and 0.2857 over the strongest competing values.