Mesh-native tokenisation enables flexible physics emulators on unstructured grids
Mesh-Native Tokenisation: Retrofitting Foundation Models to Unstructured Data
Computational Engineering, Finance, and ScienceArtificial Intelligence
Summary
Physics emulators need a way to transform raw physical data into a format computers can use. Usually, this is done with convolutional models that work well for regular grid data but fail on irregular, unstructured mesh data common in complex simulations. The authors introduce a new method called Mesh-Native Convolution that processes unstructured mesh data similarly to traditional methods, allowing models trained on one mesh to work on others without extensive retraining. This makes building physics emulators more adaptable and efficient across different data types.
What this means in practice
- •For simulation engineers: Deploy physics emulators trained on one mesh directly to various unstructured meshes with minimal retraining effort.
- •For software developers in scientific computing: Integrate a mesh-native tokenisation layer into emulators to efficiently handle data from simulations on complex geometries without switching model architectures.
Authors
Georg Maierhofer, Miles Cranmer, Payel Mukhopadhyay
Abstract
Physics foundation models are an increasingly popular approach for building versatile emulators for spatiotemporal systems. They require tokenisation layers that lift physical field data into a latent representation suitable for dynamic processing. A typical choice for this layer is a convolutional encoder, which lifts field information in a locality-aware way and can be trained very efficiently. However, this choice cannot be applied to a large class of scientific computing problems where data are represented on unstructured meshes, for example simulations on domains with curved boundaries. Such data typically require graph neural network architectures, which are more expensive to train and, in the form currently used, encode information differently from their CNN counterparts, thus limiting the portability of physics emulators between these data regimes. In this work we introduce Mesh-Native Convolution (MNC), a locality-aware tokenisation layer that encodes unstructured mesh-based data in a way that is analogous to CNNs. This approach leads to models that can be trained once on mesh data and then deployed to arbitrary unstructured meshes, even if these are irregular, with minimal retraining. We show that this tokenisation approach can be used to retrofit grid-pretrained foundation models efficiently on unstructured meshes and provide theoretical results to confirm the consistency of this approach with classical CNN encoders. We demonstrate the favourable performance of this retrofitting approach, which in many cases reaches acceptable accuracy even in zero-shot deployment, and outperforms existing baselines after only a few epochs of fine-tuning, thus providing an efficient bridge between the two data regimes.