Seismic foundation models act mainly as feature extractors not generalists
Are Seismic Foundation Models More Than Glorified Feature Extractors?
Machine LearningArtificial Intelligence
Summary
Earthquakes happen over two very different timescales—quick seismic waves and long periods between events—and current AI models trained on short time windows can't learn both at once. The authors tested different training objectives on the same neural network architecture and found that changing these targets shifts what the models are good at rather than improving overall performance. Their models work like specialized feature extractors but don’t become general-purpose seismic understanding tools. To truly capture earthquake patterns, training methods need to consider the longer timescale of when earthquakes happen, not just their wave signals.
What this means in practice
- •For seismic monitoring teams: Use few-shot trained foundation models to improve seismic wave detection and event classification with limited labeled data.
- •For earthquake hazard modelers: Incorporate foundation model features to assist in ground-motion prediction when data is scarce, complementing traditional methods in specific scenarios.
Authors
Laura Laurenti, David Liu, Brian Kulis, Men-Andrin Meier, Christopher Johnson
Abstract
Earthquake sequences are stochastic on two separated timescales: the seismic waves of any individual earthquake propagate over seconds to minutes, while the earthquakes themselves arrive as a sparse, self-exciting occurrence process, with inter-event intervals from seconds to months. Respecting both scales in one training objective is the open challenge. Foundation models (FMs) for seismic waveforms instead import their learning targets from audio and vision, where every target is computed from a single fixed-length window. Such a window holds one realization of the wave propagation (an earthquake trace), never the occurrence process that decides when earthquakes happen. We develop nine model designs sharing one Mamba encoder-decoder, trained on a fixed corpus of 3M windows, varying the self-supervised objective, quantization bottleneck and mixer; the two key comparisons are architecture-matched, varying the objective alone. Each checkpoint is evaluated frozen on three downstream tasks. Varying the target redistributes downstream skill rather than raising it, and no checkpoint wins everywhere. Adding masking pressure to our strongest classifier re-specializes it rather than making it a generalist: it gains 6.9 points of few-shot P-wave picking precision and loses 9.6 points of few-shot classification accuracy. By the standard imported from language and vision, these models are feature extractors rather than FMs. Controls on capacity, architecture, and domain proximity do not explain why skill moves rather than rises. Against task-specific baselines the advantage is regime-dependent: our checkpoints beat them few-shot on classification and ground-motion prediction, and trail them on full data. We argue the binding constraint is the target, and that settling the question requires a learning target defined across events, the timescale that makes seismicity stochastic in the first place.