Exact symplectic neural layers improve accuracy in physical system models
Integrable building blocks for symplectic neural networks: learned-type layers, fiber-preserving layers, and what a single layer can do
Machine Learning
Summary
Modeling certain physical systems requires preserving special mathematical properties called symplectic structures. The authors explore new types of neural network layers that exactly maintain these properties, unlike previous ones relying mainly on simple shear maps. They introduce learned-type, moving-plane shear, and fiber-preserving layers and show these give more accurate results in modeling complex dynamical systems. Their experiments found these new layers outperform existing symplectic networks by large margins on various test problems.
What this means in practice
- •For computational physicists: Use exact symplectic layers to improve simulation accuracy of Hamiltonian physical systems, especially for nonlinear dynamics and lattices.
- •For machine learning engineers: Design symplectic neural networks with new layer types that better preserve physical invariants, increasing model precision in scientific computing tasks.
Authors
Andrew Qing He
Abstract
Exactly symplectic neural networks are built almost exclusively from shear maps and gain expressivity only through depth. We ask what else a single explicit symplectic layer can be. Observing that every layer in use is the exact flow of a Hamiltonian that is a function of commuting integrals, we obtain three families of layers: a learned-type layer, in which every conjugate pair is rotated, squeezed or sheared by a state-dependent amount and the type is found by training; moving-plane shears, residual symplectic maps outside the known ridge-function class; and fiber-preserving layers, which combine an unconstrained point transformation with a fiber translation at the cost of one linear solve. We characterize residual symplectic layers $x \mapsto x + J\nabla H(x)$: in one degree of freedom they are exactly the shears along a line, while from two degrees of freedom on a strictly larger class exists. Conjugating a learned-type layer by a symplectic network yields an exact one-parameter group with an exactly conserved learned Hamiltonian. In experiments on sixteen systems with one to forty degrees of freedom, against SympNets, HénonNets and learned-plane shear networks, layers built on nonlinear integrals are far more accurate when the Hamiltonian contains functions of those integrals. On nonlinear Schrödinger lattices with six to forty sites, twist layers alternating with linear symplectic maps are over a hundred times more accurate than every shear-based network, and with an on-site potential of learned form three to four orders of magnitude more accurate. On chains of Tao oscillators with the step size as an input, stretch layers are 45 to 70 times more accurate than learned-plane shears and recover the Hamiltonian to 1%. On generic non-separable systems, fiber layers improve on learned-plane shears by at most a factor of two, and widening the networks does not help any architecture.