Polynomial activation networks become identifiable with high degree
Linear Independence of Polynomial Compositions with Bounded Input Degree
Machine Learning
Summary
This paper proves some long-standing math questions about how compositions of polynomials behave when their inputs have limited complexity. The authors show that neural networks using general polynomial activation functions of sufficiently large degree have a unique structure that can be identified. This means such networks can be better understood and analyzed mathematically. Their approach uses advanced geometry tools like curve deformations and fields on surfaces to reach this conclusion.
What this means in practice
- •For neural network builders: Establish network identifiability conditions for polynomial activation functions enabling more reliable neural network design and interpretation.
- •For mathematical software developers: Incorporate identifiability checks for polynomial-based neural networks to improve symbolic computation and verification tools.
A theory result. No direct application yet.
Authors
Xiaotian Chang, Yacoub Hendi
Abstract
We prove the weak composition-independence conjectures of Kohn, Marchetti, Massarenti and Mella. The proof uses deformations of rational curves and a single kernel of the linearized equation. Riemann-Roch gives a contradiction from tangent fields with controlled poles. By settling these conjectures, we prove that a neural network with a general polynomial activation function of sufficiently large degree is identifiable.