Meteorology-Driven GPT4AP: A Multi-Task Forecasting LLM for Atmospheric Air Pollution in Data-Scarce Settings
2026-03-31 • Machine Learning
Machine Learning
AI summaryⓘ
The authors developed a new forecasting model called GPT4AP that predicts air pollution levels using a version of GPT-2 combined with a special adaptation method to keep it lightweight. This model works well even with very little training data and can adapt to different locations without retraining. When tested, it was better than some existing methods in low-data settings and held its own when lots of data was available. The results suggest their approach is efficient and reliable for air pollution forecasting, especially when data is limited or comes from new regions.
air pollution forecastingGPT-2Gaussian rank-stabilized low-rank adaptation (rsLoRA)multi-task learningfew-shot learningzero-shot transfermean squared error (MSE)mean absolute error (MAE)time series forecastingdomain shift
Authors
Prasanjit Dey, Soumyabrata Dev, Bianca Schoen-Phelan
Abstract
Accurate forecasting of air pollution is important for environmental monitoring and policy support, yet data-driven models often suffer from limited generalization in regions with sparse observations. This paper presents Meteorology-Driven GPT for Air Pollution (GPT4AP), a parameter-efficient multi-task forecasting framework based on a pre-trained GPT-2 backbone and Gaussian rank-stabilized low-rank adaptation (rsLoRA). The model freezes the self-attention and feed-forward layers and adapts lightweight positional and output modules, substantially reducing the number of trainable parameters. GPT4AP is evaluated on six real-world air quality monitoring datasets under few-shot, zero-shot, and long-term forecasting settings. In the few-shot regime using 10% of the training data, GPT4AP achieves an average MSE/MAE of 0.686/0.442, outperforming DLinear (0.728/0.530) and ETSformer (0.734/0.505). In zero-shot cross-station transfer, the proposed model attains an average MSE/MAE of 0.529/0.403, demonstrating improved generalization compared with existing baselines. In long-term forecasting with full training data, GPT4AP remains competitive, achieving an average MAE of 0.429, while specialized time-series models show slightly lower errors. These results indicate that GPT4AP provides a data-efficient forecasting approach that performs robustly under limited supervision and domain shift, while maintaining competitive accuracy in data-rich settings.