Summary
Building detailed 3D models of maize plants grown in fields is important for studying plant traits and breeding, but it usually takes a lot of manual work or special training data. The authors created a way to make these 3D plant models automatically from raw 3D scans using a vision-language model to identify leaf shapes in images, which are then mapped back to the 3D data and refined into a procedural plant model. Their system worked well on hundreds of real maize plants, producing accurate and editable 3D models without needing manual annotations or species-specific training. This makes it easier to create large collections of detailed plant models for research and breeding.
What this means in practice
- •For agriculture technology teams: Generate detailed 3D models of field-grown maize to support high-throughput phenotyping and in silico breeding workflows without manual plant annotation.
- •For 3d modeling software developers: Integrate automated vision-language based annotation and geometric processing to create editable plant models from raw point cloud scans for diverse plant species.
Authors
Mozhgan Hadadi, Talukder Z. Jubery, Adarsh Krishnamurthy, Baskar Ganapathysubramanian
Abstract
Editable 3D models of field-grown crops support high-throughput phenotyping and in silico breeding trials, but building them from scanned point clouds requires organ-level segmentation and fitting. Procedural generators can turn an organ-level parameter set into an analysis-suitable 3D model, but obtaining that set requires hours of manual tuning per plant or segmentation models trained on species-specific labels. We present an automated pipeline that reconstructs procedural maize models from raw 3D point clouds without manual tuning or species-specific training data. A multimodal vision-language model (VLM) annotates leaf midlines in rendered orthographic views. Deterministic geometric algorithms back-project the annotations onto the point cloud, merge them into 3D leaves by cross-view consensus, and grow the midlines to full blades on an orientation-weighted surface graph. Measured organ parameters populate a plant descriptor for a Non-Uniform Rational B-Spline (NURBS)-based procedural model generator. Each leaf surface is then refined against its scan points by differentiable NURBS fitting. The pipeline reached a median whole-plant Chamfer distance of 5.4 mm on 100 genotypically diverse field-grown maize plants from the MaizeField3D dataset. The reconstructions were closer to the scans than those of an earlier semi-automated pipeline based on manual annotations. The pipeline recovered 1,017 of 1,023 (99.4%) curated reference leaves at an intersection-over-union of at least 0.5 without using those labels as input. These results show that VLM annotations become usable organ-level measurements when downstream geometric stages can correct them. This makes automated generation of editable 3D plant assets feasible at the scale of modern phenotyping experiments.