PrecipJEPA improves future rain forecast accuracy using motion-based modeling

PrecipJEPA: JEPA-Regularized Future-State Prediction with Motion-Source Rendering for Precipitation Nowcasting

MultimediaMachine Learning

Summary

Forecasting where and how much it will rain in the next few hours is hard because rain patterns move and change intensity in complex ways. The authors propose PrecipJEPA, a new method that learns from radar images over time to better predict future rain by understanding both motion and intensity changes. It uses two paths: one that predicts future radar images and another that learns from past data by filling in missing parts, helping the model get better at understanding rain patterns. Tests show it significantly improves accuracy for predicting heavy rainfall over three hours compared to previous methods.

What this means in practice

  • For weather forecasting teams: Improve short-term radar-based rainfall forecasts by incorporating motion and intensity modeling for better heavy rain prediction.
  • For disaster response coordinators: Use enhanced precipitation nowcasting to support early warning systems for localized flooding events up to three hours ahead.

Authors

Yufeng Zhu, Dan Niu, Qiliang Wu, Weiwei Huang, Yixiao Liang, Yongchao Feng, Chunlei Shi

Abstract

Long-term precipitation nowcasting requires modeling radar-echo evolution while preserving localized high-intensity structures. Recent radar-specific studies motivate location-aware prediction and separating echo displacement from intensity change. However existing encoders learn historical representations mainly from final forecast errors. We propose PrecipJEPA, which couples a structured forecasting path with an auxiliary path that enriches its encoder from observed radar history. In the forecasting path, an online encoder first converts the observations into spatiotemporal tokens. The Task-Driven Future-State Predictor (TFP) combines these tokens with a recent-dynamics summary and spatiotemporal queries to construct future radar states. The Parallel Motion-Source Renderer (PMSR) decodes these states into motion and source-sink fields that transform the latest observation into future frames. During joint training, the History-Masked JEPA (H-JEPA) operates on the auxiliary path to predict masked historical features from visible context, directly supervising the same online encoder from the observed sequence. Experiments on SEVIR and MeteoNet show that PrecipJEPA improves highest-threshold CSI by 118.6% and 35.1%, respectively, over the strongest baselines, while maintaining the highest mean CSI throughout the 3-hour forecast.