Efficient algorithms find and use exact symmetries in data learning
Efficient Learning and Symmetry Discovery under Exact Invariances
Machine Learning
Summary
Finding patterns that stay the same under transformations, called symmetries, helps machines learn better from data. The authors show how to efficiently train models that perfectly respect these symmetries, even when the symmetries come from infinite groups. They also develop a method to discover unknown symmetries from data and use them to improve learning, all with guarantees that work in reasonable time. This advances understanding of how to build smarter learning systems that exploit geometric structure.
What this means in practice
- •For machine learning engineers: Enable building regression models that respect exact symmetries in data efficiently, improving prediction quality on structured inputs.
- •For computer vision developers: Discover unknown geometric symmetries in image data to improve model accuracy without prior knowledge of transformations.
Authors
Ashkan Soleymani, Behrooz Tahmasebi, Patrick Jaillet, Stefanie Jegelka
Abstract
Learning with group invariances is central to many scientific and geometric learning problems, yet its computational foundations remain poorly understood. Even for classical supervised regression settings, it has been unclear whether one can efficiently compute a regression function that is exactly invariant to a given group action. Recent work showed that exact invariance can be enforced in polynomial time when the underlying group is finite and known, but left open the cases of infinite groups and unknown symmetries. In this paper, we resolve both challenges. First, we present the first polynomial-time algorithm for learning with exact group invariances that applies uniformly to finite and infinite groups. The runtime is polynomial in the data dimension and sample size, and independent of the group, while achieving strong generalization guarantees. This provides a computational explanation for the empirical success of invariant and equivariant methods in geometric machine learning and partially answers a recent open question in the literature. Second, we study learning in the symmetry discovery setting, where the invariance group is unknown. Focusing on the subgroup lattice of a finite group, we show that exact symmetries can be identified from data and exploited for learning in polynomial time. For regression over finite-dimensional feature spaces, our algorithm provably recovers the underlying symmetry, matches the minimax-optimal sample complexity of the known-symmetry setting, and runs in time polynomial in the data dimension and sample size. Our analysis relies on tools from random Cayley graphs and expander theory, which may be of independent interest.