Physically guided neural networks improve interpretability and data use
On architectural choices for interpretability and thermodynamic consistency in Physically Recurrent Neural Networks in the low-data regime
Machine Learning
Summary
It can be hard to train smart computer models with very little data while keeping their results easy to understand. The authors studied how changing parts inside a specific type of neural network helps reveal meaningful hidden information about physical processes. They also found a way to design the network so it better follows basic physics rules, like energy conservation, which makes the model more reliable and easier to train with less data. This approach helps connect tiny-scale details with bigger scale behavior without needing more data or bigger models.
What this means in practice
- •For materials scientists: Use physically consistent neural models to infer microscopic material states from limited experimental data efficiently.
- •For engineering simulation teams: Incorporate robust interpretable physics-based neural networks to improve multi-scale simulations without increasing data or computational cost.
Authors
M. A. Maia, K. A. Meyer, A. M. C. M. van Gils, I. B. C. M. Rocha, F. P. van der Meer
Abstract
In this paper, we unravel the effect of different decoder architectures on the interpretability of the latent space of the Physically Recurrent Neural Network. Particular emphasis is given to a new weight normalization constraint, which acts as a regularization technique and enables robust training in the low-data regime. A brief visual exploration illustrates how these changes impact the latent space and how the fictitious stress can align with the true state of the RVE without explicit training. Reaping the benefits of a meaningful latent space, a case study illustrates how information from the microscopic level can be retrieved and incorporated into a multi-task approach that does not require extra parameters or larger training sets. Another key contribution shows that a specific architectural choice can naturally lead to a thermodynamically consistent formulation. By enforcing an adjoint encoder-decoder structure with positive scalar contributions, this modification ensures energy consistency across scales and non-negative dissipation, leading to even lower training requirements. This alternative completes the study on interpretability, inductive bias, and thermodynamic consistency, and demonstrates that data efficiency can be improved with careful architectural choices rooted in the underlying physics.