Vision transformers outperform CNNs in identifying orchid genera from photos
Vision Transformers versus convolutional neural networks for fine-grained orchid genus identification in a species-rich, data-poor flora: a controlled benchmark on the Orchidaceae of New Guinea
Computer Vision and Pattern RecognitionMachine Learning
Summary
Identifying orchid species from photos is hard because many species have few pictures available. The authors tested two kinds of AI models, vision transformers and convolutional neural networks, to see which is better at recognizing orchid genera from a limited number of images. They found that vision transformers trained with a certain self-supervised method did best, even when data was scarce. Their system first guesses the orchid genus, then finds similar species images, and is available as a web tool for practical use.
What this means in practice
- •For wildlife conservation teams: Deploy a tool to identify orchid genera from limited photo collections to support biodiversity monitoring in species-rich, data-poor regions.
- •For mobile app developers: Build plant identification apps that first classify genus then retrieve species images, enhancing accuracy with vision transformer backbones.
Authors
Reza Saputra, Diah Harnoni Apriyanti, André Schuiteman, Kurt Metzger, Ashley Field, Katharina Nargar, William Edwards
Abstract
New Guinea is the world's richest island flora (~2,856 orchid species), yet most species are represented by only a handful of photographs, far fewer than direct species-level classification requires. Methods for fine-grained identification in such species-rich, data-poor floras are needed, and it remains unclear which backbone architecture and pretraining strategy best support them. We built a two-stage system that first predicts the genus of a query photograph, then retrieves visually similar reference images of candidate species using FAISS. We compared four pretrained backbones -- two Vision Transformers (ViTs; DINOv2, BioCLIP 2) and two CNNs (ConvNeXt V2-L, EfficientNetV2-L) -- fine-tuned under an identical protocol on a fixed, species-stratified partition of 16,701 photographs spanning 120 genera and 1,350 species, assessing accuracy, calibration, error structure, species retrieval, and open-set detection of novel genera. DINOv2 attained the best genus performance (macro top-1 66.9%, 95% CI 63.7-70.6; global top-1 88.9%); both ViTs outranked both CNNs, and general-purpose self-supervised pretraining (DINOv2) outperformed domain-matched biological pretraining (BioCLIP 2) by 7.1 points of macro top-1. Errors concentrated on two abundant genera acting as error attractors. DINOv2 embeddings achieved species Recall@5 of 86.6% and genus Recall@5 of 98.7%; temperature scaling reduced every backbone's Expected Calibration Error to about 0.03; and a distance-based open-set gate flagged unseen genera (mean AUROC 0.958). A self-supervised Vision-Transformer backbone combined with embedding retrieval is an effective, deployable strategy for fine-grained identification in species-rich, data-poor floras. The system is released as an open web application (the New Guinea Orchid Identifier), offering a practical template for other hyperdiverse, under-documented taxa.