Annotation-free 3d plant organ labeling achieves high accuracy without training
Zero-Shot 3D Plant Organ Segmentation with SAM3 and Semantic NeRFs
Computer Vision and Pattern Recognition
Summary
Segmenting 3D images of different plant parts usually needs lots of labeled examples or specialized models. The authors created a new system that only requires photos taken from multiple angles and a list of plant part names, with no manual labeling or special tuning. Their method combines a text-guided image segmenter with a 3D reconstruction technique to produce accurate 3D labels. Tested on several plant types, it achieves accuracy close to approaches that use perfect labels.
What this means in practice
- •For agriculture technology teams: Create accurate 3D models of plant organs without manual labeling to speed up phenotyping in crop research and breeding.
- •For garden automation developers: Automatically identify and label plant parts in 3D for smart gardening systems that customize care by organ.
Authors
Andreas Gilson, Laura Hennig, Peter Pietrzyk
Abstract
Accurate 3D plant organ segmentation is fundamental to automated phenotyping. Existing approaches rely on annotated training data or species-specific model configurations. We present an annotation-free pipeline for 3D plant organ segmentation, combining text-prompted SAM3 segmentation with semantic neural radiance fields (NeRFs). Given only multi-view RGB images and a list of class names, our zero-shot pipeline produces semantically labeled 3D point clouds without manual annotation, per-species fine-tuning, or domain-specific preprocessing. Multi-view NeRF fusion acts as effective implicit consensus mechanism that lifts imperfect per-frame masks into accurate 3D labels. On a controlled Begonia maculata testbed the SAM3 pipeline achieves 92.6% mIoU, reaching 95.9% of the oracle upper bound established with perfect ground-truth masks. The pipeline was further evaluated on a new dataset spanning ten diverse plant point clouds reaching an average 0.856 mIoU, with leaf and pot IoU above 0.91 and 0.90 for every species, respectively. These results demonstrate that annotation-free 3D plant organ segmentation is now feasible and approaching the range of supervised methods.