Equivariant neural networks explained with graded polynomial theory
Graded Representation Theory of Equivariant Neural Networks
Machine Learning
Summary
Neural networks often use nonlinear activation functions, which help create complex interactions that simpler linear maps can’t capture. This paper explores how these nonlinear effects can be understood using a tool called Gaussian degree decomposition, extending the idea of polynomial degree to these more complex cases. The authors show that for a certain kind of neural network layer respecting symmetries, the nonlinear behavior can be separated into parts related to the network’s structure, the choice of coordinates, and the activation functions. This helps clarify what limits the network’s behavior based on symmetry and activation.
What this means in practice
- •For machine learning engineers: Design equivariant neural network layers with a clearer understanding of how activations affect symmetry and interactions in model behavior.
- •For robotics control developers: Improve control algorithms that rely on symmetry properties by accounting for nonlinear activation influences described in the paper.
A theory result. No direct application yet.
Authors
Mani Shayestehfar
Abstract
Nonlinear activations can create equivariant interactions between irreducible representations that linear maps cannot. We use the Gaussian degree decomposition to extend ordinary polynomial degree to such nonlinear maps, and prove that for a fixed coordinatewise equivariant layer each degree factors into a polynomial determined by the linear maps and a scalar determined by the activation. This separates three distinct obstructions, coming from symmetry, coordinates, and activation.