Towards 3D Scene Understanding of Gas Plumes in LWIR Hyperspectral Images Using Neural Radiance Fields
2026-03-05 • Computer Vision and Pattern Recognition
Computer Vision and Pattern Recognition
AI summaryⓘ
The authors studied how to use special 3D models called Neural Radiance Fields (NeRFs) to combine several thermal images taken from different angles into a detailed 3D picture. They focused on longwave infrared images to detect gas leaks, specifically a sulfur hexafluoride plume, in a simulated environment. By improving existing NeRF methods with fewer images and a new way to measure errors, their model created accurate 3D representations and helped detect gas plumes effectively. This shows potential for better analyzing scenes with limited data in infrared imaging.
Hyperspectral imagingLongwave infrared (LWIR)Neural Radiance Fields (NeRF)3D scene reconstructionGas plume detectionMip-NeRFDIRSIG simulationPeak Signal-to-Noise Ratio (PSNR)Adaptive weighted MSE lossAdaptive coherence estimator (ACE)
Authors
Scout Jarman, Zigfried Hampel-Arias, Adra Carr, Kevin R. Moon
Abstract
Hyperspectral images (HSI) have many applications, ranging from environmental monitoring to national security, and can be used for material detection and identification. Longwave infrared (LWIR) HSI can be used for gas plume detection and analysis. Oftentimes, only a few images of a scene of interest are available and are analyzed individually. The ability to combine information from multiple images into a single, cohesive representation could enhance analysis by providing more context on the scene's geometry and spectral properties. Neural radiance fields (NeRFs) create a latent neural representation of volumetric scene properties that enable novel-view rendering and geometry reconstruction, offering a promising avenue for hyperspectral 3D scene reconstruction. We explore the possibility of using NeRFs to create 3D scene reconstructions from LWIR HSI and demonstrate that the model can be used for the basic downstream analysis task of gas plume detection. The physics-based DIRSIG software suite was used to generate a synthetic multi-view LWIR HSI dataset of a simple facility with a strong sulfur hexafluoride gas plume. Our method, built on the standard Mip-NeRF architecture, combines state-of-the-art methods for hyperspectral NeRFs and sparse-view NeRFs, along with a novel adaptive weighted MSE loss. Our final NeRF method requires around 50% fewer training images than the standard Mip-NeRF and achieves an average PSNR of 39.8 dB with as few as 30 training images. Gas plume detection applied to NeRF-rendered test images using the adaptive coherence estimator achieves an average AUC of 0.821 when compared with detection masks generated from ground-truth test images.