Napping boosts spiking neural networks by balancing accuracy and simplicity
Exploring napping paradigm for Recurrent Spiking Neural Networks
Machine Learning
Summary
Spiking Neural Networks try to mimic how brains learn, but they usually keep their rules strict and simple. This paper studies a new approach inspired by sleep, where the network ‘naps’ by softly scaling its connections and adding some randomness. The researchers found that this napping can keep the network as accurate as usual methods but also help it find clearer patterns in data. This trade-off might make the networks better at understanding structure, although it takes more computing power.
What this means in practice
- •For neuromorphic hardware developers: Improve spiking neural networks' ability to represent structured data by incorporating biologically inspired napping for better feature separation.
- •For machine learning engineers: Train recurrent spiking neural networks with napping to balance model stability and complexity in tasks requiring unsupervised feature learning.
Authors
Andreas Massey, Stefano Nichele, Aliaksandr Hubin
Abstract
Biological organisms minimize free energy by balancing two competing demands on their internal world model: it must be accurate enough to predict sensory input, yet simple enough to generalize beyond it. Two mechanisms regulate this balance offline: sleep reduces complexity through gradual synaptic downscaling, while stochastic noise attenuates precision, relaxing the constraint sensory input imposes on synaptic reorganization. Engineered Spiking Neural Networks (SNNs) leave this balance unaddressed, favoring instantaneous, noiseless weight normalization instead. This paper investigates the hypothesis that a biologically inspired micro-sleep paradigm, napping -- combining proportional weight scaling with continuous stochastic membrane activity -- can replicate the stability of normalization while shedding model complexity. We evaluate this in an unsupervised recurrent SNN trained via trace-based spike-timing-dependent plasticity (STDP) on Gabor-preprocessed MNIST. We tune napping across three regularization regimes by sweeping its duration and membrane noise level, then compare the best configuration against weight normalization. Across all three regimes, well-tuned napping matches the accuracy of normalization: accuracy peaks at brief durations and low noise, then declines monotonically as either grows. Clustering diverges, with the strongest geometric separation arising at longer durations and higher noise -- the two terms of free energy pulling apart, accuracy rewarding data fit and structure rewarding the simpler representation that gradual, noisy downscaling induces. This gain carries a simulation cost normalization avoids, so napping is most compelling where representational structure, rather than raw classification efficiency, is the priority.