Learning stability requires frequent and clear feedback signals
On the stability of learning
Machine Learning
Summary
Learning often depends on having useful information available all the time, but in some systems like the brain, this information can be rare and noisy. The authors show that if this learning signal is often missing or noisy, the learning process can become unstable even if it seems to be going in the right direction on average. They analyze simple network models and find conditions that ensure learning stays stable, showing that just correct feedback isn't enough; it must also be frequent and clear. This insight adds new understanding to how learning can be reliable in both artificial and biological systems.
What this means in practice
- •For machine learning engineers: Design training systems by ensuring gradient or feedback signals are frequent enough and low-noise to maintain stable learning.
- •For neuromorphic hardware designers: Configure feedback delivery in hardware models of brain-like networks to meet stability conditions for reliable learning under intermittent signals.
A theory result. No direct application yet.
Authors
John J. Vastola
Abstract
Learning algorithms are usually analyzed under the assumption that informative gradients are continuously available. But this is not always true---learning signals in biological neural networks, for example, may be sparse, local, and only intermittently available at any given synapse. In this work, we show that if gradient information is noisy and only intermittently available, learning may be unstable. In particular, even if average learning dynamics converge to the correct optimum, variance in learning may be asymptotically infinite. We explore this point in a number of analytically tractable settings, including single-layer and multi-layer linear networks, and derive explicit conditions under which learning is stable. These conditions complement, but are conceptually distinct from, existing conditions related to the stability of stochastic gradient descent. Our work demonstrates that unbiased credit assignment is not enough for reliable learning; sufficiently frequent and informative feedback is also required, which yields an additional constraint on models of biologically plausible learning.