Visibility-aware method improves 3D reconstruction in dense vegetation
Visibility-Aware Reference Generation for Benchmarking Visual 3D Reconstruction in Vegetation-Rich Environments
Computer Vision and Pattern RecognitionRobotics
Summary
Evaluating 3D models of environments with lots of plants is hard because it’s expensive to scan everything perfectly. The authors created a new way to use existing camera images and LiDAR scans to generate accurate reference depth maps that take visibility into account. Their method keeps more detailed points than some other approaches, making it easier to compare depth and 3D reconstruction methods. They found that fast feedforward techniques create dense but sometimes messy 3D models, while traditional methods give clearer but sparser results. The authors plan to share their tools and data to help others test similar methods.
What this means in practice
- •For environmental surveyors: Generate accurate depth references from partial LiDAR and images for evaluating 3D models of vegetated outdoor areas.
- •For robotics engineers: Assess and improve feedforward and SfM depth estimation methods for navigation in complex, plant-rich outdoor environments.
Authors
Zhipeng Zhong, Matthew Coombes, Cunjia Liu
Abstract
Reliable evaluation of visual 3D reconstruction and depth estimation in vegetation-rich outdoor environments is limited by the scarcity of geometric ground-truth data and the cost of complete scanner surveys. We present a LiDARbased reference-generation pipeline for evaluating the zero-shot performance of feed-forward visual-geometry models alongside conventional structure-from-motion (SfM) methods. The pipeline reuses camera images, LiDAR scans, and reference poses from existing multi-sensor localization datasets to generate per-view depth maps and visible reference point clouds. To address visibility errors in accumulated LiDAR scans, we introduce the LargeDepth-Gap Visibility Score (LDGVS) for point-wise visibility filtering and automatic scale selection in Hidden Point Removal(HPR). Experiments on TartanAirV2 and TartanGround show that, compared with Poisson surface reconstruction, which is an approach suited to completed scan surveys, the combined HPR and LDGVS filtering retains approximately three times the point density at comparable absolute relative error. We also apply the pipeline to the BotanicGarden dataset to systematically compare depth estimation and 3D reconstruction methods. Feedforward methods produce dense depth predictions efficiently, but their 3D reconstructions can contain duplicated structures. Conventional SfM generally produces more coherent geometry, but its reconstructions remain sparse and more prone to failure in challenging vegetated scenes. The benchmark toolkit and datasets will be released.