Papers for

neural network builders

Papers whose findings have a practical use for this group, as judged from the abstract. Open a paper to read what it means in practice.

Polynomial activation networks become identifiable with high degree

Linear Independence of Polynomial Compositions with Bounded Input Degree

Abstract: We prove the weak composition-independence conjectures of Kohn, Marchetti, Massarenti and Mella. The proof uses deformations of rational curves and a single kernel of the linearized equation. Riemann-Roch gives a contradiction from tangent fields with controlled poles. By settling these conjectures, we prove that a neural network with a general polynomial activation function of sufficiently large degree is identifiable.

Wed 14 OctMachine Learning

The gist

This paper proves some long-standing math questions about how compositions of polynomials behave when their inputs have limited complexity. The authors show that neural networks using general polynomial activation functions of sufficiently large degree have a unique structure that can be identified. This means such networks can be better understood and analyzed mathematically. Their approach uses advanced geometry tools like curve deformations and fields on surfaces to reach this conclusion.

Open → 2610.20825v1