Vision transformers estimate sunlight from limited sky images

Vision Transformers for Estimating Irradiance Using Data Scarce Sky Images

Artificial IntelligenceComputer Vision and Pattern RecognitionMachine Learning

Summary

Measuring sunlight accurately is important for solar power but hard in places without many instruments. The authors developed a new method using vision transformers, a type of AI model, to estimate sunlight from a small number of sky pictures. They include calculations about the sun’s position and cloud patterns to help the model work better with little data. This method was tested in a cloudy climate and gave results close to those from larger data-based models, making it useful for places with limited measurement tools.

What this means in practice

  • For solar energy operators: Estimate solar irradiance accurately in regions lacking extensive ground instruments using limited sky images and physics-informed models.
  • For weather forecasting teams: Improve solar radiation forecasts by integrating irradiance estimates from few sky images when ground data is unavailable.

Tested on one dataset.

Authors

David Hamlyn, Sunny Chaudhary, Tasmiat Rahman

Abstract

Accurate estimation of diffuse horizontal irradiance (DHI) is critical for optimising photovoltaic system performance and energy forecasting yet remains challenging in regions lacking comprehensive ground-based instrumentation. Recent advancements using Vision Transformers (ViTs) trained on extensive sky image datasets have shown promise in replacing costly irradiance measurement equipment, but the scarcity of long-term, high-quality sky imagery significantly restricts practical implementation. Addressing this critical gap, this study proposes a novel dual-framework approach designed for data-scarce scenarios. First, calculated atmospheric parameters, including extraterrestrial irradiance and cyclic time encodings, are integrated to represent sky conditions without utilising any instrumentation. Next, a sequential pipeline initially predicts synthetic global horizontal irradiance (GHI) and uses it as a feature, to refine DHI estimation. Finally, a dual-parallel architecture simultaneously processes raw and overlay-enhanced fisheye sky images. Overlays are generated through unsupervised, physics-informed cloud segmentation to highlight dynamic sky features. Empirical validation is performed using data from the Chilbolton Observatory, chosen for its temperate climate and frequent cloud variability. To simulate data-scarce conditions, models are trained on a single month (e.g., January) and evaluated across a temporally disjoint, full-year test set. Under this setup, the sequential and dual-parallel frameworks achieve RMSE values within 2-3 W/m^2 and 1-6 W/m^2, respectively, of a state-of-the-art ViT trained on the complete dataset. By combining physics-informed modelling with unsupervised segmentation, the proposed method provides a scalable and cost-effective solution for DHI estimation, advancing solar resource assessment in data-constrained environments.