Machine learning improves iterative methods for solving equations
An overview of machine learning-enhanced iterative methods for systems of linear and nonlinear equations
Machine Learning
Summary
Solving systems of linear and nonlinear equations is important in science and engineering but can be slow and tricky with traditional methods. The authors review how combining machine learning with classical iterative techniques can make these solvers more efficient and reliable. This approach keeps the steps understandable while improving performance. They discuss current methods, challenges, and ideas for future improvements.
What this means in practice
- •For computational engineers: Use hybrid ML-iterative methods to speed up solving engineering simulation equations that involve large complex systems.
- •For financial modelers: Improve convergence in nonlinear equation solvers used for risk and pricing models with ML-enhanced iterative methods.
A survey. It maps existing work.
Authors
Yuhuang Meng, Jing Zhao, Alexander Heinlein
Abstract
Systems of equations arise in a wide range of scientific and engineering applications. The present work focuses on solvers for general systems of equations, including but not limited to those arising from partial differential equations. These systems can be broadly categorized into linear and nonlinear problems. For large linear systems, iterative solvers are generally preferred over direct methods due to the latter's superlinear growth of computational costs. Although convergence theory is well-developed under certain assumptions on the coefficient matrix, many classes of systems still pose open challenges. These difficulties become even more severe for systems of nonlinear equations, where nonlinear solvers typically rely on repeated linearization. For example, Newton's method may even converge quadratically near the solution; it can also converge slowly or diverge when the initial guess is not chosen appropriately. A wide range of solvers with diverse variants and hyperparameter settings exists, and the development of efficient and robust iterative methods remains an active area of research. Recently, machine learning (ML) techniques have been applied to enhance the efficiency of classical iterative methods while preserving their interpretability and reliability. We refer to these ML-enhanced iterative methods as hybrid iterative methods, in the sense that they combine classical iterative methods with ML. This paper provides a comprehensive overview of state-of-the-art approaches to constructing hybrid iterative methods for systems of both linear and nonlinear equations, while also discussing open challenges and outlining potential directions for future research.