Reinforcement learning enables seismic imaging without gradient access
DeepWaveRL: Self-Supervised Full Waveform Inversion via Reinforcement Learning
Machine Learning
Summary
Estimating underground properties like rock velocity using seismic waves is a complex task. The authors present a novel approach that treats this estimation problem as a game where a computer learns by receiving feedback on its guesses, without needing detailed directions on how to improve each guess. They use reinforcement learning to guide the computer to make better predictions just by looking at how well simulated seismic data matches real data. Despite the challenge of learning from only a simple score, their method performs comparably to traditional techniques that rely on more detailed information.
What this means in practice
- •For geophysical exploration teams: Improve underground velocity map estimation from seismic data even without gradient calculations from simulation models.
- •For industrial nondestructive testing engineers: Apply reinforcement learning based inversion methods to estimate material properties where simulation gradients are inaccessible.
Authors
Peng Jin, Yinan Feng, Yinpeng Chen, Youzuo Lin, Sharon Huang
Abstract
Full Waveform Inversion (FWI) is a fundamental inverse problem in geophysics to estimate subsurface properties, such as velocity, from seismic measurements. Recently, self-supervised learning methods remove the need for ground-truth velocity maps during training by leveraging a forward operator to simulate seismic data from predictions and backpropagate the gradients of reconstruction loss through it. In this work, we consider a more challenging setting without accessing such gradients. To this end, we incorporate reinforcement learning into self-supervised FWI and propose a framework named DeepWaveRL. Specifically, we reformulate FWI as a policy learning problem, where the policy network predicts velocity maps as actions, and a forward operator is used solely for simulation. A global scalar reward is computed based on data misfit for each prediction and used to weight policy gradients. Thus, learning proceeds without accessing the gradients from the forward operator. Given the indirect and coarse supervision signals, we further identify key strategies to stabilize learning and mitigate the spatial credit assignment bottleneck. Experimental results show that, despite relying entirely on a single scalar reward, DeepWaveRL achieves competitive performance compared to supervised and self-supervised baselines. These findings suggest that our framework offers a flexible and complementary learning paradigm for the FWI research community.