Neural Acquisition & Representation of Subsurface Scattering
2026-06-01 • Computer Vision and Pattern Recognition
Computer Vision and Pattern Recognition
AI summaryⓘ
The authors developed a method to understand how light scatters just beneath the surface of different objects by learning detailed light patterns on each point of the object's surface. They use 3D scanning combined with a special neural network (U-Net CNN) to analyze data captured by a projector and camera setup. This lets them recreate realistic lighting effects from any angle and pattern. Their tests show that the method's predictions closely match real photographed results and can also apply to new, unseen materials.
sub-surface scattering3D scanningU-Net CNNstereo projector-camera setupphase-shifted profilometry (PSP)pixel footprintlight transportrelightingneural networkcomputer vision
Authors
Arjun Majumdar, Raphael Braun, Hendrik Lensch
Abstract
We present a method to acquire and estimate the sub-surface scattering properties of light transport at a highly detailed level by learning the pixel footprint response at each point on the object surface. The reconstruction leverages 3D scanning techniques as input to a U-Net CNN. A stereo projector-camera setup using phase-shifted profilometry (PSP) patterns efficiently captures the data for a variety of scattering objects. Reconstructing dense pixel footprints allows for relighting with arbitrary high-resolution projector patterns. The final output is a relit color image. Qualitative and quantitative comparison against illuminated real-world captured images demonstrate that the predicted footprints are almost identical to the actual responses. The same model is trained for multiple views across multiple objects such that the learned representations can be used to generalize to unseen sub-surface scattering materials as well.