Spiking neural networks improve robustness with new learning method
Robust Representation Learning for Spiking Neural Networks based on Conditional Information Bottleneck
Artificial Intelligence
Summary
Spiking neural networks (SNNs) are energy-efficient models that mimic brain activity but can be easily disturbed by noise or attacks. The authors found that this vulnerability comes from how these networks represent input data. They developed a new training approach called SCIB that guides the network to learn more robust and organized internal representations, improving resistance to noise and attacks while reducing energy use. Experiments show that SCIB works better than standard methods on common image recognition tasks even under challenging conditions.
What this means in practice
- •For neuromorphic hardware engineers: Enhance noise and attack robustness of spiking neural networks deployed in neuromorphic chips to improve real-world reliability.
- •For embedded systems developers: Reduce energy consumption during inference on low-power devices by using robust SNN models trained with SCIB.
Authors
Yuli Zha, Wang Fan, Yixiao Dai, Lingjie Wang, Bang Hu, Wei Cao, Fengzhe Zhang, Fan Zhang
Abstract
Spiking Neural Networks (SNNs) have shown great potential for low-power neuromorphic computing due to their event-driven processing and spatiotemporal sparsity, but they remain vulnerable to input perturbations, hardware-intrinsic noise, and adversarial attacks in real-world deployment. We attribute this vulnerability partly to input-specific hidden representations learned by standard SNNs trained with conventional cross-entropy loss, as our information-plane analysis empirically shows that substantial input-related information can still be retained after label-predictive information becomes stable. To address this issue, we propose SCIB, a simple yet effective robust learning objective for SNNs based on the Conditional Information Bottleneck. Different from conventional variational IB methods, SCIB replaces the mutual information compression term $I(X;S)$ with the conditional mutual information $I(X;S|Y)$ and derives a tractable variational upper bound. Moreover, it introduces a learnable class-conditional prior instead of a label-agnostic isotropic Gaussian prior, improving the expressiveness of variational compression and encouraging class-structured representations when combined with the supervised classification objective. Systematic experiments on Fashion-MNIST, CIFAR-10, and CIFAR-100 under external environmental noise, neuromorphic hardware noise, and adversarial attacks show that SCIB improves robustness over standard SNNs and IB-based baselines under most evaluated noise conditions, improves adversarial accuracy under the evaluated attacks without using adversarial training, and reduces the SynOps-based single-sample inference energy consumption by 24.9% relative to an unconstrained SNN baseline.