3D point splatting improves 3D view creation from mmWave radar data
3D Point Splatting for mmWave Radar Novel View Synthesis
Computer Vision and Pattern RecognitionGraphicsMachine Learning
Summary
Creating accurate 3D views from millimeter-wave radar data is challenging because the technology requires complex calculations involving waves and materials. The authors present a new method called 3D Point Splatting that treats radar signals more precisely and can handle different types of radar outputs without retraining. Their approach uses a mathematical model based on how radar waves bounce off materials and then spreads this information into range data efficiently. This method performs better than previous techniques on outdoor scenes and trains quickly on modern graphics hardware.
millimeter-wave radarnovel view synthesis3D point splattingdifferentiable renderingradar equationcomplex-valued datarange-azimuth imagesFourier transformpoint spread functionmaterial modeling
Authors
Adnan Armouti, Yixuan Gao, Rajalakshmi Nandakumar
Abstract
Solving novel view synthesis (NVS) for millimeter-wave (mmWave) radar requires a renderer that is physically faithful, complex-valued, and multi-viewpoint-tractable. No prior method achieves these three properties simultaneously. Differentiable Monte Carlo (MC) ray tracers implement the radar forward model directly with explicit material modeling and complex outputs, but do not scale to the multi-view optimization NVS demands. Optical-NVS ports of NeRF, hash grids, and 3D Gaussians train fast but discard phase and replace explicit material modeling with opaque learned features, restricting them to power-only range-azimuth (RA) magnitudes. We propose 3D Point Splatting (3DPS), the first differentiable point renderer for radar, derived directly from the standard solid-angle form of the radar equation. Each oriented 3D point carries an ITU-R P.2040 material model, evaluated in closed form, with the resulting complex phasor splatted into range bins through a precomputed point spread function (PSF). The complex-valued output makes the renderer product-agnostic. The same optimized scene yields analog-to-digital converter (ADC), complex range profile (CRP), and RA outputs through standard fast Fourier transform (FFT) pipelines without retraining for each format. On six outdoor ColoRadar scenes, 3DPS reaches 0.587 mean Pearson correlation on held-out RA images. This is between 1.7x and 5.2x the three optical-NVS baselines (RadarSplat, Radar Fields, DART). Training takes approximately 3 minutes per scene on a single RTX 4090.