Frozen encoders improve radar image understanding with less data
Preserving the Manifold: Frozen Expert Alignment for Label-Efficient SAR Scene Understanding
Computer Vision and Pattern Recognition
Summary
Understanding radar images with text is hard because radar data looks very different from normal pictures, and there are not many labeled radar-text examples. The authors found that changing pre-trained models to work together messes up the radar data's important features. Instead, they keep both models fixed and only add a small connection between them, which keeps the radar information intact. This approach works better than retraining models and needs much less labeled data for tasks like recognizing radar scenes.
What this means in practice
- •For remote sensing analysts: Use frozen encoders with a learned bridge to classify radar images with fewer labeled examples and better accuracy.
- •For defense intelligence teams: Improve radar imagery scene understanding without needing large annotated datasets by preserving radar physics in pretrained models.
Authors
Eli A. Underwood, I. Taylor McKechnie, Leslie M. Collins
Abstract
Cross-modal alignment of synthetic aperture radar (SAR) imagery to natural language is constrained by two distinct failures. The first is modality mismatch: text encoders pre-trained on optical imagery, such as CLIP, cannot be efficiently fine-tuned on SAR, because active microwave backscatter is physically incompatible with passive optical reflectance and the available SAR caption datasets are orders of magnitude too small to bridge this gap without substantially degrading the learned semantic manifold. The second is physical forgetting: SAR encoders pre-trained on large-scale self-supervised objectives encode radar physics that are progressively erased when the encoder is fine-tuned for text alignment on limited caption data. A single design principle addresses both failures: keep both encoders frozen and learn only a constrained, structure-preserving bridge. Our Frozen Expert framework aligns a frozen CROMA SAR encoder with a frozen CLIP text encoder via a lightweight constrained residual bridge. We formally define physical forgetting via Procrustes disparity and show our bridge preserves SAR geometry significantly better than fine-tuning (disparity: 0.210 vs. 0.351). We validate generality across multiple SAR backbones and text encoders. On zero-shot SAR scene classification, our approach outperforms fine-tuned baselines using up to 8x less training data: DFC2020 (+10.3% R@1, +15.5% mAP), EuroSAT-SAR (+15.7% R@1, +12.2% mAP), BigEarthNet-MM (+14.7% R@1). These results establish frozen manifold alignment as a principled, data-efficient alternative to fine-tuning in high-disparity cross-modal settings.