Cnn image classifiers rely on background context in agricultural pictures
Evaluating Contextual Bias in CNN Image Classification: Evidence from Agricultural Benchmark Datasets
Computer Vision and Pattern Recognition
Summary
Convolutional neural networks (CNNs) are usually tested by how well they recognize objects in images, assuming they focus on the objects themselves. The authors found that CNNs trained on farm-related pictures also use clues from the background rather than just the main object to make their guesses. This background influence varies by dataset and model and is sometimes due to uneven data but can persist even when data are balanced. Their work shows that relying on surrounding background is common for CNN image classifiers, not just in farming images but in other areas too.
What this means in practice
- •For agricultural technology developers: Improve robustness of CNN models for plant and crop identification by accounting for background bias in agricultural image datasets.
- •For medical imaging engineers: Assess and mitigate contextual bias in CNN-based pathology image classifiers by applying the paper’s evaluation framework.
Authors
Abhilekha Dalal, Michael Okonoda, Eder Martinez, Lior Shamir
Abstract
Convolutional neural networks (CNNs) are typically evaluated using held-out classification accuracy, an approach that presupposes predictions are based primarily on the intended object of interest rather than incidental surrounding context. We test this assumption in CNN-based agricultural image classification by comparing model performance on original images with performance on background-dominated patches extracted from the same images across eight publicly available agricultural benchmark datasets and four widely used CNN architectures. Background-dominated patches were classified above dataset-specific random chance for six of the eight datasets, and substantially above chance for four of them, indicating that contextual information contributes to model predictions for the majority of datasets evaluated. For these four datasets, we further evaluated whether this behavior reflected genuine class-discriminative information or was primarily attributable to class imbalance using macro-averaged precision, recall, and F1 together with class-balanced test subsets. The results show that contextual reliance does not admit a single explanation: class imbalance accounts for a substantial portion of the observed signal for some datasets and architectures, whereas above-chance contextual classification persists after balancing for others. Together with previous evidence from curated object recognition and cancer pathology imaging, these findings support the growing view that contextual bias is a recurring characteristic of CNN-based image classification rather than a phenomenon confined to a single application domain. More broadly, this work provides a systematic framework for quantifying contextual bias across heterogeneous image datasets by combining dataset-specific random-chance baselines, contextual bias categorization, macro-averaged evaluation, and class-balanced robustness analysis.