Hierarchical rank-evolving representation for physics-informed neural networks
2026-08-10 • Machine Learning
Machine Learning
AI summaryⓘ
The authors introduce a new method called hierarchical rank-evolving (HRE) representation to improve how physics-informed neural networks handle complex multivariate functions. Unlike older methods that require manually choosing fixed ranks, HRE automatically figures out the best ranks during its process, making the models easier to use and more accurate. They tested their approach on a variety of challenging physics problems and found that it consistently performed better than existing techniques. This makes HRE a promising tool for solving practical scientific problems using neural networks.
Tensor decompositionPhysics-informed neural networks (PINNs)Multivariate functionsRank adaptationHierarchical representationHelmholtz equationPoisson equationKlein-Gordon equationNavier-Stokes equationTensor networks
Authors
Ruoyang Su, Xi-Le Zhao, Kun Li, Liang Li
Abstract
Recently, tensor-based physics-informed neural networks (T-PINNs) have received increasing attention. However, existing T-PINNs still face a fundamental challenge: they mainly rely on pre-specified low-rank tensor decompositions with manually tuned ranks, which limits their ability to capture the underlying structures of multivariate solution functions and hinders their practical deployment. To address this challenge, we propose a hierarchical rank-evolving (abbreviated as HRE) representation for multivariate functions, which endows us to faithfully capture the underlying structure of the targeted multivariate function accompanying with automatic rank determination. Concretely, in the hierarchical design of HRE representation, the target multivariate function is decomposed as a small-scale inner tensor with a set of univariate functions along each mode, where a customized tensor network decomposition can be readily deployed to capture the underlying structure of the small-scale inner tensor. In HRE representation, the crucial hyperparameters, ranks, can be adaptively revealed during the decomposition, freeing us from manual rank tuning and making HRE practically applicable to real-world problems. Besides, we build the HRE-PINNs correspondingly. Extensive numerical experiments, including high-dimensional static problems (Helmholtz equation and Poisson equation), nonlinear time-dependent problems (Klein-Gordon equation), and complex fluid-dynamics problems (flow mixing equation and Navier-Stokes equation), demonstrate that HRE-PINNs consistently outperform existing state-of-the-art approaches in terms of accuracy.