Broadband Hyperspectral 3D Imaging using Dispersed Structured Light
2026-05-25 • Computer Vision and Pattern Recognition
Computer Vision and Pattern Recognition
AI summaryⓘ
The authors developed a new imaging method that captures detailed color and 3D shape information across a wide range of light wavelengths, from visible to short-wavelength infrared. Their system uses two cameras and a single device that breaks light into components, allowing it to see surface details and hidden material properties. This helps reveal things like hidden features beneath surfaces and differences in materials that look the same to the eye. They tested their method on real scenes and achieved accurate results for both color and depth measurements.
hyperspectral imaging3D imagingvisible spectrumshort-wavelength infrared (SWIR)structured lightspectrographreflectancedepth reconstructionmetameric materialsstereo camera setup
Authors
Suhyun Shin, Yunseong Moon, Ryota Maeda, David Lindell, Kyros Kutulacos, Seung-Hwan Baek
Abstract
Hyperspectral 3D imaging enables the capture of dense spectral information and scene geometry but has traditionally been confined to narrow spectral windows, typically the visible range. In this work, we introduce a broadband hyperspectral 3D imaging (BH3D) method to extend this capability across the full visible-near-infrared and short-wavelength infrared (SWIR) spectrum (450-1500 nm). This broad coverage is critical as it captures complementary physical cues: visible wavelengths reveal surface appearance, while SWIR bands provide insight into subsurface properties and material composition. However, realizing BH3D is challenging due to fundamental sensor constraints between visible-spectrum silicon and SWIR-spectrum InGaAs sensors, which necessitate complex multi-spectrograph designs. Here we propose a single-spectrograph BH3D system, using a stereo setup comprising visible and SWIR cameras, that reconstructs dense broadband hyperspectral reflectance together with accurate 3D geometry. Our key idea is to extend dispersed structured light to the broadband regime using a single spectrograph. We model the image formation of broadband dispersed structured light, and estimate hyperspectral reflectance and depth. We validate our approach on diverse real-world scenes, demonstrating accurate reconstruction with a mean spectral angle mapper of 0.13 rad, root mean square error of 0.03, and mean depth error of 4.5 mm. We further demonstrate identifying metameric materials, performing imaging through opaque layers, uncovering hidden features on banknotes, and revealing blood vessels.