Efficient learning of exact symmetries in data with unknown groups
Efficient Learning and Symmetry Discovery under Exact Invariances
Machine Learning
Summary
Figuring out how to make computer learning models respect exact symmetries in data is important but often very hard. The authors developed a fast method that works whether the symmetry group is small, huge, or even unknown beforehand. Their approach can quickly find these symmetries from data and use them to improve learning, doing this with strong guarantees on accuracy and efficiency. They also use some advanced math about special graphs to make their method work well. This helps explain why symmetry-based learning techniques work well in many real-world problems.
group invariancemachine learningregressionsymmetry discoverypolynomial-time algorithmsfinite groupsinfinite groupsgeneralization guaranteesCayley graphsexpander theory
Authors
Ashkan Soleymani, Behrooz Tahmasebi, Patrick Jaillet, Stefanie Jegelka
Abstract
Learning with group invariances is central to many scientific and geometric learning problems, yet its computational foundations remain poorly understood. Even for classical supervised regression settings, it has been unclear whether one can efficiently compute a regression function that is exactly invariant to a given group action. Recent work showed that exact invariance can be enforced in polynomial time when the underlying group is finite and known, but left open the cases of infinite groups and unknown symmetries. In this paper, we resolve both challenges. First, we present the first polynomial-time algorithm for learning with exact group invariances that applies uniformly to finite and infinite groups. The runtime is polynomial in the data dimension and sample size, and independent of the group, while achieving strong generalization guarantees. This provides a computational explanation for the empirical success of invariant and equivariant methods in geometric machine learning and partially answers a recent open question in the literature. Second, we study learning in the symmetry discovery setting, where the invariance group is unknown. Focusing on the subgroup lattice of a finite group, we show that exact symmetries can be identified from data and exploited for learning in polynomial time. For regression over finite-dimensional feature spaces, our algorithm provably recovers the underlying symmetry, matches the minimax-optimal sample complexity of the known-symmetry setting, and runs in time polynomial in the data dimension and sample size. Our analysis relies on tools from random Cayley graphs and expander theory, which may be of independent interest.