How Merge-Tolerant Are Vision Transformers for Wheat Phenotyping?
2026-08-24 • Computer Vision and Pattern Recognition
Computer Vision and Pattern Recognition
AI summaryⓘ
The authors studied how to use Vision Transformer models for different wheat-related tasks like figuring out growth stages, counting wheat heads, and identifying parts of the plant. They focused on a method called token merging, which can speed up the models without needing to train them again. Their tests showed that merging tokens works well for classifying growth stages but is harder for detection and segmentation because these tasks need more detail. They also found that real speed improvements depend on the hardware and software used, so you can’t just trust token counts alone.
Vision TransformerToken MergingWheat PhenotypingGrowth-Stage ClassificationWheat-Head DetectionOrgan SegmentationHigh-ThroughputEdge InferenceGPU MemoryAttention Mechanism
Authors
Simon Ravé, Pejman Rasti, David Rousseau
Abstract
Vision-based wheat phenotyping requires repeated measurements under deployment constraints, from growth-stage recognition to wheat-head counting and organ segmentation. Plain Vision Transformers (ViTs) provide a common architecture for these tasks, but quadratic attention limits high-throughput and edge inference. Training-free token merging is attractive because it can be inserted into trained models without retraining. We provide a systematic benchmark of ToMe and Mutual Pair Merging across growth-stage classification, wheat-head detection, and wheat-organ segmentation, measuring task quality, throughput, token count, and peak GPU memory, with additional Raspberry Pi 5 measurements. The benchmark reveals a clear hierarchy: classification is highly merge-tolerant, while detection and segmentation are constrained by repeated instances, thin organs, dense boundaries, reconstruction, and runtime overhead. Optimized attention backends can erase apparent speedups, so deployment value must be profiled on the target runtime rather than inferred from token count.