Single-photon lidar sees hidden scenes behind translucent obstacles
Resolving Mixed Single-Photon LiDAR Returns for Foreground-View and Hidden Scene Reconstruction
Computer Vision and Pattern Recognition
Summary
Sometimes cameras or sensors can’t see through shiny screens or protective covers because the signals get mixed from the cover and what’s behind it. This paper shows a way to use a special kind of lidar sensor, called single-photon lidar, that measures the timing of light pulses very precisely to detect both the cover and what’s behind it. The authors made a smart computer program that guesses whether the light signal comes from one or two objects and then reconstructs the 3D shapes of both. They also created a new dataset to test their method and found it works better than older methods at seeing hidden objects.
What this means in practice
- •For robotic inspection teams: Detect hidden features behind protective covers or partially transparent barriers using enhanced lidar 3D reconstruction.
- •For autonomous vehicle developers: Improve perception systems to detect objects behind foggy or partially see-through obstacles using single-photon lidar data.
Authors
Ziting Wen, Runrong Deng, Zili Zhang, Haitao Zheng, Yuecong Xu, Xiaoqiang Ren, Guodong Shi, Kemi Ding
Abstract
Partially transmissive screens and protective covers are common in robotic inspection, but they create mixed LiDAR returns from both the foreground material and the scene behind it. Conventional peak-based LiDAR usually discards weak hidden returns, while single-photon LiDAR records time-resolved histograms that preserve attenuated and overlapping echoes. However, existing transient reconstruction methods typically fit a single scene representation to the measured waveform. Under occlusion, weak or nearby foreground--hidden echoes can form a broad peak or subtle shoulder. Because such waveforms can also be explained by a displaced single surface or a thick density distribution, accurate transient fitting does not necessarily imply correct geometry. We propose a state-aware framework for foreground-view and hidden scene reconstruction from occluded single-photon histograms. For each ray, we estimate local echo evidence, identifying no reliable surface evidence, single-return evidence, or two returns. The inferred echo state routes supervision for a two-head neural field: all rays constrain waveform reconstruction, while reliable anchors provide geometry localization. We also introduce a real paired single-photon LiDAR occlusion dataset with occluded and clean captures at fixed poses. Experiments on a real dataset show improved hidden scene depth and point-cloud accuracy over baselines. Our results demonstrate single-photon layered reconstruction as a practical route for 3D perception through partially transmissive occluders.