Box-supervised model segments tree crowns in sparse aerial images
Latent Commonality Expectation-Maximisation for Box-supervised Tree Crown Instance Segmentation
Computer Vision and Pattern Recognition
Summary
Counting individual tree crowns from aerial photos helps track forests, but current methods struggle with dry areas where trees are spread out and look different. The authors developed LACE, a new model that learns to find tree crowns using only simple box labels around trees, instead of detailed drawings. By using clever image features from a pretrained model and a special technique called expectation-maximisation, LACE matches or beats older methods that require lots of detailed labels. This makes tree counting and monitoring possible in places with fewer labeled examples and simpler annotations.
What this means in practice
- •For forest monitoring teams: Build tree crown maps in dry, sparse forests using only bounding boxes instead of detailed polygon labels, reducing annotation effort significantly.
- •For urban planners: Generate accurate tree crown segmentations from aerial images for sparse urban greenery without needing expensive detailed labels.
Authors
Thomas Pitts, Kunqi Li, Bin Liang
Abstract
Individual tree crown segmentation from aerial imagery underpins tree-level carbon accounting, biodiversity, and restoration monitoring at landscape scale. However, existing models are predominantly trained on dense canopy forest imagery and degrade in savannah and drylands, where tree crowns are sparse, of variable appearance, and underrepresented in annotated benchmarks. These models also typically depend on costly polygon annotations. We introduce LACE (LAtent Commonality Expectation-maximisation), a box-supervised instance segmentation model, evaluated on 0.1 m/px aerial RGB tree crown imagery. LACE uses a frozen DINOv3-web ViT-L/16 encoder, applied at four spatial offsets and interlaced into a denser feature grid, with a lightweight CenterNet-style detection head trained solely on bounding boxes. We use expectation-maximisation to separate recurring appearance, the "treeness", within bounding boxes from surroundings. On the OAM-TCD benchmark test set, LACE reaches a mask AP$_{50}$ of $0.663 \pm 0.001$ (3 seeds) trained on 900 box-annotated images and without mask annotations, above the 0.626 scored by Restor's released mask-supervised Mask R-CNN, which was trained on the full ~4.2k image set. On a sparse-canopy holdout set, mask AP$_{50}$ rises to $0.691$ versus $0.612$ for Detectree2, a mask-supervised baseline. On NeonTreeEvaluation, using the official evaluation code, LACE reaches $0.728 \pm 0.003$ F1@0.4 (5 seeds) from 23,424 hand-annotated RGB boxes alone, matching the authors' DeepForest model's published 0.719, using under 0.1% of its training annotations and none of its LiDAR-derived 30M-crown pretraining set. By leveraging frozen self-supervised features, LACE matches or surpasses fully-supervised specialist baselines from boxes alone, removing the need for polygon annotation in tree crown instance segmentation for sparse-canopy environments where labelled data is scarce.