AI summaryⓘ
The authors address the problem of finding small objects hidden in plants, which is hard because there are few labeled pictures and models trained on one place don't work well elsewhere. Instead of using labeled data from a different location, they create new labeled training images by starting with a few unlabeled photos from the target site. Their method builds a rough 3D scene of the plants, adds virtual objects to get labels automatically, then uses a special image process to make the scene look more realistic. Training detectors on these synthetic images worked as well or better than training on a larger set of real images from another site, showing their approach helps adapt to new environments without manual labels.
labeled imagessmall object detectionvegetation3D scene synthesisvision-language modeldiffusion modeldomain adaptationbounding boxessegmentation maskshumanitarian demining
Authors
Mario Malizia, Marnix Enting, Rob Haelterman, Ken Hasselmann
Abstract
Labeled images of small objects hidden in vegetation are scarce, and detectors trained on them generalize poorly across sites. Rather than reusing labels collected at another site, we synthesize labeled training images from a handful of unlabeled photographs of the deployment site itself. A vision--language model generates a coarse 3D vegetation scene from one photograph; placing 3D object meshes in the scene yields bounding boxes, segmentation masks, and per-instance occlusion directly from the scene geometry, without manual annotation. A lightweight adapter fine-tuned on the photographs conditions a diffusion pass that re-textures the renders, and a graded mask-lock sets how much diffusion may touch the object itself. In our runs this grade was the most influential curation choice: lightly diffusing the object improves minority-class recall over fully protecting its pixels, while unrestricted diffusion dissolves it. Trained on these images, a standard detector matched or exceeded its counterpart trained on a larger labeled dataset of real images from a different site, consistently across seeds on a humanitarian-demining benchmark; the comparison is thus unsupervised site adaptation from a handful of photographs against conventional cross-site label reuse. In our ablations the gains were largely insensitive to the photograph and crop budgets, and in-domain accuracy did not predict cross-site performance.