(MPO)$^2$: Multivariate Polynomial Optimization based on Matrix Product Operators
Machine Learning
Summary
The authors focus on improving how machines learn complex relationships between inputs and outputs using polynomial models. Traditional polynomial models get too big and complicated as they try to capture more details. The authors propose a new method called (MPO)² that keeps these models smaller and more flexible by using a special kind of mathematical structure called Matrix Product Operators. Their approach works well for different learning tasks and overcomes some limitations of earlier methods.
Authors
Niccolò Ciolli, Anders Vestergaard Nørskov, Michael Kastoryano, Petr Taborsky, Morten Mørup
Abstract
Central to machine learning and signal processing is the ability to perform universal function approximation and learn complex input-output relationships from limited numbers of observations. Multivariate polynomial models offer a natural way to express such relationships through multiplicative feature interactions, but their coefficient tensors grow exponentially in size with the polynomial degree. Existing tensorized polynomial models reduce this cost, yet canonical polyadic decompositions have rank-limited expressivity, and tensor train formulations are feature order dependent. We introduce Multivariate Polynomial Optimization based on Matrix Product Operators (MPO)$^2$, a framework that combines learned MPO feature embeddings with compact polynomial weight tensors. This yields feature order independent polynomial representations that can incorporate structured operators such as projections, convolutions, and masks for weight tensor symmetries. Across regression and classification benchmarks, (MPO)$^2$ improves over existing tensor decomposition based polynomial models and provides a flexible alternative for efficient polynomial function approximation.