Microfacet-driven neural method improves 3d surface reconstruction
MicroSDF: Microfacet-Driven Hybrid Neural SDFs for Mixed-Reflectance Surface Reconstruction
Computer Vision and Pattern Recognition
Summary
Creating accurate 3D models of objects is hard when those objects have both shiny (reflective) and dull (non-reflective) parts. The researchers developed a new approach called MicroSDF that uses detailed ideas about how light reflects off tiny surface details to better capture both types of surfaces together. Their method separates the way colors appear based on reflectivity and blends them more realistically. Tests show it makes high-quality 3D reconstructions across different datasets that include shiny and matte objects.
What this means in practice
- •For computer vision teams: Improve 3D scanning accuracy for scenes with mixed shiny and matte surfaces using physically based neural reconstructions.
- •For augmented reality developers: Create more realistic 3D models of real environments by better handling reflective surfaces during reconstruction.
Authors
Lejia Ye, Yuhua Xu, Yulan Guo, Lian Xu, Xiaohu Zhang
Abstract
Accurate 3D reconstruction in real-world environments remains a significant challenge due to the coexistence of reflective and non-reflective surfaces, which pose distinct modeling demands. Existing methods often treat these surface types separately, limiting their generalizability and physical plausibility. To bridge this gap, we propose MicroSDF, a novel neural implicit framework that facilitates geometry and reflectance modeling through microfacet theory. Our approach incorporates three core innovations: (i) a microfacet-guided geometry model that extracts multi-scale surface normals (macroscopic and microfacet) from a signed distance field (SDF), regularized by a proposed microfacet normal consistency loss to enforce physically plausible surface orientations; (ii) an enhanced dual-branch color model, where the specular branch leverages the microfacet normals to model high-frequency reflectance, and the vanilla branch, unlike prior works, uses reflection direction (instead of viewing direction) to better model diffuse and low-frequency specular components; and (iii) a detection-guided color blending strategy that adaptively fuses the color outputs based on reflection priors, providing more physically intuitive blending than implicitly learned blending weights. Combined with a tailored multi-stage optimization scheme, the proposed MicroSDF achieves robust and high-fidelity reconstruction across reflective and non-reflective surfaces. Extensive experiments on DTU, Shiny Blender, Ref-NeRF, and DeepVoxels datasets demonstrate state-of-the-art performance, establishing a new direction for physically grounded neural reconstruction.