Graphcast model reveals hidden measures of atmospheric river strength
Mechanistic Interpretability of Atmospheric Rivers in GraphCast
Machine LearningArtificial Intelligence
Summary
Weather AI models can make great forecasts, but it's unclear how they think about weather internally. The authors trained special tools called sparse autoencoders on the GraphCast model to find what ideas it forms inside, focusing on atmospheric rivers, which are big flows of moisture in the air. They found that GraphCast calculates a measure of river strength called integrated vapor transport as an internal concept, even though it’s not directly told to do so. This method helps explain what the AI really uses to make predictions and could help check if AI understands weather as climates change.
What this means in practice
- •For weather model developers: Identify stable internal variables in AI weather models to improve model interpretability and reliability.
- •For climate impact analysts: Use internal variables uncovered by the method to assess whether model concepts remain valid under climate change.
Authors
Madelyn Mathai, Timothy B. Higgins, Kevin M. Grise, Chirag Agarwal, Antonios Mamalakis
Abstract
While AI weather models now rival operational forecasts, how they represent the atmosphere internally remains an open question: feature attribution reveals which input patterns matter, not what the model computes or how it combines information internally. We train sparse autoencoders (SAEs) on GraphCast to uncover its learned concepts, using atmospheric rivers as our phenomenon of focus. Both standard and Matryoshka SAEs show GraphCast computes atmospheric river intensity, measured by integrated vapor transport (IVT), as a stable internal variable, despite IVT being neither an input nor a target. In contrast to the unstructured concept retrieval of the standard SAE, the Matryoshka SAE orders concepts by importance and exposes their relations. Atmospheric river concepts persist across depth and direct interventions confirm causality. This method offers a way to find internal variables and determine which of them the model actually relies on, which is a prerequisite for asking whether those variables remain meaningful as the phenomenon changes under a warming climate.