Spiking neural networks save energy by pausing inactive neurons
The Dormant Spiking Neuron: A State-Driven Mechanism for Efficient Spiking Neural Networks
Computer Vision and Pattern Recognition
Summary
Spiking neural networks (SNNs) are a kind of artificial intelligence that can save energy by only working when they receive input. However, many neurons still use energy even when they are not active. The authors found that nearly half the neurons can be in a deeply inactive state yet still consume power. They created a system called Dormant Spiking Neuron (DSN) that lets these quiet neurons 'sleep' and stop wasting energy, while still keeping the network accurate. This method reduces computational energy use by up to 70% and improves accuracy on image recognition tasks.
What this means in practice
- •For embedded system engineers: Implement energy-saving dormant neuron mechanisms in neuromorphic hardware to reduce power consumption while maintaining performance.
- •For mobile ai developers: Design efficient spiking neural network models for mobile devices that dynamically reduce redundant neuron computations and extend battery life.
Authors
Ruichen Ma, Yicai Chen, Pujun Zhou, Yue Zuo, Chongyang Li, Shuang Liu, Shaogang Hu, Guanchao Qiao
Abstract
Spiking neural networks (SNNs) offer a compelling pathway toward energy-efficient artificial intelligence, fundamentally anchored in presynaptic input-driven sparsity. However, a substantial yet chronically overlooked source of computational overhead persists: the continuous subthreshold dynamics of postsynaptic neurons. Quantitative analysis reveals that up to 44% of neurons in an ImageNet-trained ResNet-18 model reside in a deeply inhibited state, imposing a persistent computational burden despite having a negligible probability of firing. To address this inefficiency, the Dormant Spiking Neuron (DSN) model is introduced. By incorporating a state-driven dynamic gate, the DSN allows a neuron to enter a dormant state when its membrane potential falls below a predefined negative threshold, dynamically pruning redundant computations at the instance level. Crucially, this dormancy mechanism is directly integrated into the neuronal dynamics and trained end-to-end via backpropagation through time using surrogate gradients, enabling the network to actively learn a robust dormancy policy that adaptively preserves critical information flow. Extensive experiments across static and neuromorphic datasets demonstrate that the DSN framework consistently reduces computational energy by 40% to over 70%. Notably, on the large-scale ImageNet dataset, the DSN outperforms state-of-the-art static pruning techniques at comparable reported cost reductions and simultaneously improves the corresponding baseline SNN accuracy by 1.34%. By establishing a trainable, state-driven sparsity paradigm orthogonal to conventional input-driven methods, the DSN unlocks synergistic efficiency gains, presenting an effective approach for advancing neuromorphic computing.