Semantic Reconstruction and 3-D Detection via Learned Multi-Pair Fusion in RF Imaging
2026-08-24 • Computer Vision and Pattern Recognition
Computer Vision and Pattern Recognition
AI summaryⓘ
The authors studied a way to create 3D images from radio signals that change depending on where the transmitter and receiver are placed. They used different mathematical methods to combine these signals and trained a special neural network (a 3D U-Net) to label each tiny part of the image as belonging to certain object types. Their approach also groups these parts into whole objects and works better in noisy conditions compared to traditional methods. Additionally, they taught their model to recognize when an object doesn’t belong to any known class, avoiding wrong guesses.
Multistatic imagingAnisotropyInverse problemBack-projectionLASSO3D U-NetSemantic segmentationInstance segmentationOutlier exposureSignal-to-noise ratio
Authors
Amir Rezaei, Wen-Xin Pan, Giuseppe Caire
Abstract
We consider a multistatic radio-frequency imaging problem with anisotropy, in which the reflection from a point depends on the positions of the transmit (Tx) and receive (Rx) arrays. The goal is to label the voxels of a field of view by a finite set of semantic classes and to group them into object instances. For the image formation of each Tx--Rx pair we apply a standard inverse-problem solver, and we feed the resulting per-pair reconstructions into a trained three-dimensional (3-D) U-Net that performs the fusion implicitly and the per-voxel classification explicitly. On a controlled, under-determined multistatic setup, we consider the following image formation methods: back-projection (BP) and the least absolute shrinkage and selection operator (LASSO) from a single deterministic snapshot, and incoherent BP and group-LASSO from multiple fading snapshots. For each imaging method we train a separate U-Net that fuses the six Tx--Rx pairs (its input channels) and assigns each voxel a probability vector over the classes. Taking the most probable class gives a labeled volume---the semantic reconstruction. Object instances and their oriented bounding boxes then follow by geometric post-processing (clustering and principal-component analysis). Across a wide range of signal-to-noise ratio, the semantic reconstruction (scored against ground truth by segmentation intersection-over-union) and the resulting 3-D detection degrade far more gracefully than the classical intensity reconstruction: the detection in particular stays reliable well into noise levels at which that reconstruction has dissolved. Because real scenes contain objects of classes the network was not trained on, we add an explicit unknown class trained by outlier exposure, which labels held-out novel objects as unknown instead of mislabeling them as a known class by reconstructed shape.