Transformer neural network improves sampling for XY model lattices
Sampling the XY model with an attention-based neural network
Machine Learning
Summary
The XY model is a physics system used to study magnetism and phase transitions on a grid. Sampling its behavior accurately is important but hard. The authors developed a new neural network using transformers and attention techniques that can better generate sample configurations of this system. Their method works well even for large grids and matches traditional methods while being more efficient.
What this means in practice
- •For computational physicists: Generate precise samples of two-dimensional continuous spin systems near critical transitions efficiently.
- •For machine learning engineers: Incorporate locality-informed attention mechanisms to improve sampling in neural networks for continuous spatial data.
Authors
Piotr Białas, Christian D. Morales, Tomasz Stebel, Dawid Zapolski
Abstract
We present a transformer-based variational autoregressive network for sampling lattice systems with continuous degrees of freedom and apply it to the two-dimensional XY model near the Berezinskii-Kosterlitz-Thouless transition. The architecture is combined with physics-informed ingredients: locality-informed attention and approximate probabilities. The resulting sampler significantly outperforms existing neural samplers. We train the model on lattices up to $96\times96$, verify with a mode-dropping estimator that no parts of the configuration space are missed, and reproduce the energy, magnetization and helicity modulus obtained with the traditional Markov Chain Monte Carlo algorithm.