Multi-task learning method improves geospatial vision-language models
Gradient-Guided Decoupled Adaptation for Geospatial Vision-Language Models
Computer Vision and Pattern RecognitionArtificial IntelligenceComputation and LanguageMachine Learning
Summary
Geospatial vision-language models help computers understand maps and satellite images along with text. The authors found these models struggle because different tasks affect each other when trained together. They introduced a new way to separate tasks based on how their learning signals interact and train them in a smarter order. This approach helps the models perform better on multiple geospatial tasks at once.
What this means in practice
- •For geospatial analytics teams: Improve performance of models that analyze satellite images combined with text-based inputs for urban planning or environmental monitoring.
- •For natural language processing developers: Enhance training methods for language-vision models that handle multiple diverse tasks simultaneously, reducing task conflicts.
Authors
Dongdong Wang, Deepak Balakrishnan, Ravi Srinivasan, Shenhao Wang
Abstract
Existing geospatial vision-language models (Geo-VLMs) typically optimize diverse geospatial tasks through a unified multi-task adaptation paradigm without explicitly accounting for the heterogeneous optimization characteristics. Our empirical observations reveal heterogeneous gradient characteristics across tasks, including vision-language differences, intra-branch gradient relationships, and task interference, which hinder effective multi-task optimization. Motivated by these observations, we propose Gradient-Guided Decoupled Adaptation (G2DA), a gradient-aware optimization framework for multi-task Geo-VLM learning. G2DA first partitions tasks into vision- and language-centric groups through gradient-guided cross-modal decoupling. It then constructs modality-specific curricula based on task gradient similarity and employs bidirectional rehearsal to mitigate the recency effects introduced by sequential optimization. We evaluate G2DA on three Geo-VLM benchmarks using six InternVL3 and Qwen3.5-VL variants, along with GeoChat and GeoLLaVA. Across all 24 benchmark-model combinations, G2DA consistently outperforms representative baselines, improving over the strongest competitor by 3.08, 4.30, and 2.81 percentage points on UrBench-MCQ, XLRS-Bench-Lite, and VRS-Bench-VQA, respectively. These results demonstrate the effectiveness of gradient-guided task organization for Geo-VLM adaptation.