AgriScope enables pixel-level understanding of agricultural images
AgriScope: Pixel-Grounded Multimodal Understanding for Agricultural Images
Computer Vision and Pattern Recognition
Summary
Farmers and agricultural experts need detailed information from images to identify plant diseases, pests, and crop types accurately. Existing AI models mainly only generate text descriptions without linking details to specific parts of an image. The authors designed AgriScope, a system that can recognize and highlight precise image regions related to agricultural features, and also hold multi-turn conversations about them. They created a huge dataset called AgriGround with millions of examples to teach the system to connect words to exact pixels in images. This approach helps machines understand complex farm images in a more detailed and interactive way.
What this means in practice
- •For agricultural technology developers: Build AI systems that pinpoint and describe plant diseases and pests directly on farm images for precise diagnosis and monitoring.
- •For environmental monitoring teams: Use pixel-level grounded models to track crop health and species identification in satellite or drone imagery for ecosystem management.
Authors
Abderrahmene Boudiaf, Mohamad Alanssari, Irfan Hussain, Sajid Javed
Abstract
Agricultural image understanding requires fine-grained recognition of plant diseases, pests, crop structures, and botanical species under complex real-world conditions. Despite recent advances in Multimodal Large Language Models (MLLMs), existing models remain limited to text-only outputs and lack pixel-level visual grounding capabilities. In this work, we introduce AgriScope, a unified pixel-grounded multimodal framework for agricultural image understanding. AgriScope jointly supports image-level, region-level, and pixel-level understanding within a unified framework, enabling tasks such as grounded caption generation, referring expression segmentation, and multi-turn multimodal interaction for agricultural imagery. AgriScope integrates biologically specialized semantic representations with dense spatial grounding through biological-semantic encoding, dense spatial representations, and pixel decoding. To support large-scale grounded learning, we introduce AgriGround, a large-scale pixel-grounded agricultural multimodal instruction-tuning dataset containing over 500K images and 11M instruction-following samples spanning plant disease analysis, crop and weed identification, insect pest recognition, and fine-grained botanical understanding. AgriGround is constructed through a multi-stage automatic annotation pipeline that integrates multimodal caption generation, phrase-level grounding, segmentation mask generation, and task-oriented instruction synthesis to produce densely grounded supervision. Extensive experiments across multiple agricultural vision-language tasks demonstrate the effectiveness of AgriScope in pixel-grounded multimodal understanding, establishing a strong benchmark for agricultural vision-language learning and visual grounding. The dataset and code will be made publicly available at (https://github.com/boudiafA/AgriScope)