JEDI improves cropland mapping using smaller satellite image models
JEDI: JEPA-to-Edge Distillation for Efficient Cropland Segmentation from Satellite Imagery
Computer Vision and Pattern RecognitionMachine Learning
Summary
High-quality models that analyze satellite images to identify farmland are usually too large and slow for use directly on satellites or nearby devices. The authors created JEDI, a method that teaches a smaller, faster model to copy the knowledge of a large one in two steps, keeping the smaller model accurate while much lighter. This approach works better than older ways of copying knowledge and almost matches the large model’s accuracy on a farmland mapping task. It shows it’s possible to run good farmland segmentation with much smaller models that can be deployed at the edge.
What this means in practice
- •For satellite system engineers: Deploy smaller, efficient segmentation models on satellites for real-time cropland mapping without large computational resources.
- •For field monitoring teams: Use lightweight models on edge devices near farms to quickly analyze satellite images for cropland segmentation with near-large-model accuracy.
Tested on one dataset.
Authors
Kishor Kumar Bhaumik, Nicolas Roque dos Santos, Jia Chen, Evangelos E. Papalexakis
Abstract
Large vision models provide useful representations for remote-sensing segmentation but are often too expensive for deployment at the satellite or field edge. Existing feature-level distillation methods also tend to assume similar teacher and student architectures and often stop feature alignment when task training begins. We introduce JEDI (JEPA-to-Edge Distillation), a two-stage framework that transfers representations from a large I-JEPA Vision Transformer teacher to a compact SegFormer student. First, JEDI aligns the student's terminal representation with the teacher's token space using cross-architecture projection and spatial alignment. It then jointly optimizes supervised segmentation, temperature-scaled response distillation, and persistent feature alignment throughout task adaptation. On CalCROP21, JEDI-B0 achieves 68.0 mean Intersection-over-Union (mIoU) with 4.04M parameters, improving over the standalone student by 16.0 points and coming within 2.0 points of the 70.0 mIoU achieved by the 639M-parameter teacher. We evaluate SegFormer B0, B1, and B2 students with 4.04M, 14.33M, and 28M parameters, respectively. Across all three variants, JEDI consistently outperforms response-, structure-, channel-, and relational-distillation baselines under the same teacher-student setting. These results show that persistent representation alignment is especially valuable under aggressive compression, substantially reducing model size and computation while preserving segmentation performance.