Deep learning improves detection of empty spaces inside forest canopies
Predicting Void Points in Self-Occluding Forest Volumes
Computer Vision and Pattern Recognition
Summary
Estimating the amount of biomass in forests using images is tricky because leaves and branches block parts of each other, creating hidden gaps. The authors found that standard methods mistake empty spaces for vegetation, causing overestimates. They trained a new model that explicitly learns to recognize empty air between trees, by labeling simulated forest scenes with both visible parts and voids. This approach helps drones and robots better understand forest structure by more accurately identifying where there is no vegetation.
What this means in practice
- •For drone operators: Improve forest biomass estimates by using models that detect empty spaces under dense canopies from drone-captured images.
- •For remote sensing engineers: Develop better occupancy maps of forests with machine learning models trained on labeled simulated data, enabling more accurate monitoring.
Authors
Benjamin Bachmayr, Mohamed Youssef, Oliver Bimber
Abstract
Synthetic Aperture Imaging (SAI) enables drones to observe forest canopies by warping and averaging multiple displaced views at a common depth. However, occlusion is suppressed rather than eliminated, causing poorly observed regions to appear occupied and leading to systematic biomass overestimation. The Deep Forest project cannot distinguish reconstruction artifacts from real vegetation because its training data relies on measured reflectance points, excluding empty space and preventing the network from learning explicit occupancy boundaries. This thesis addresses the problem by modifying training labels rather than the imaging process. Each simulated scene is rendered twice: as a 9x9 grid of top-down captures for focal-stack generation and as 550 depth-clipped slabs derived from scene geometry to provide occupancy labels, explicitly representing void space as a negative class. A Gazebo pipeline generates procedural forest stands containing 80-200 trees across three species. A pretrained Pix2Pix model translates rendered RGB images into a synthetic near-infrared band, and Light Field Rendering produces aligned focal stacks of 550 planes across three channels. Four architectures were evaluated on a held-out scene. The plain 2D U-Net achieved an IoU of 0.6290, outperforming the volumetric 3D U-Net (0.5707). Established reconstruction methods performed substantially worse: COLMAP achieved 0.3446 and DepthAnything3 0.2959, both after their respective optimal dilation sweeps. An RGB-input variant of the volumetric architecture achieved the highest IoU of 0.6514 without dilation or per-scene calibration. These results highlight the limitations of conventional reconstruction methods, which recover visible surfaces but struggle to infer occupancy within closed canopies.