Papers for

disaster monitoring centers

Papers whose findings have a practical use for this group, as judged from the abstract. Open a paper to read what it means in practice.

Latent space filtering improves forecasting with sparse noisy data

LD-EnFF: Latent-Dynamics Ensemble Flow Filtering for Data Assimilation with Sparse Observations

Abstract: Data assimilation combines model forecasts with noisy, incomplete observations to estimate the evolving state of a dynamical system. Existing methods face two compounding challenges: high-dimensional nonlinear dynamics make repeated forward simulation computationally expensive, while sparse observations provide limited direct information about the full state. To address these challenges, we propose the Latent-Dynamics Ensemble Flow Filter (LD-EnFF), a sequential Bayesian filtering framework that performs both forecast propagation and filtering updates in a compact latent space. LD-EnFF combines a latent dynamics surrogate for ensemble propagation with a variational autoencoder (VAE)-based observation model that evaluates a state-dependent observation likelihood in latent space. At each assimilation step, an ensemble filtering update based on flow matching uses the forecast ensemble and this likelihood to generate posterior samples, jointly updating latent states and uncertain parameters. This design avoids repeated full-state simulation during forecasting and full-field reconstruction during likelihood evaluation. LD-EnFF substantially outperforms a broad range of data assimilation algorithms on benchmarks spanning Kolmogorov flow, tsunami propagation, and atmospheric modeling, all featuring complex dynamics and sparse, noisy observations.

Fri 2 OctMachine Learning
The gist
Forecasting how complex systems change is hard because simulations take a lot of time and observations are limited and noisy. The authors developed a new method called LD-EnFF that works in a simpler, compressed version of the system, called a latent space, to speed up forecasting and make better guesses. LD-EnFF uses a special neural network to understand how observations relate to this latent space and updates its predictions more efficiently. Tests show it outperforms many existing methods on problems like fluid flow, tsunamis, and weather with sparse and noisy data.
Open → 2610.04034v1