Spiking neural network pruning boosts efficiency on hardware accelerators

SparseForge: Beyond Spike Sparsity toward Hardware-Exploitable Neuron Pruning for Efficient SNN Acceleration

Hardware Architecture

Summary

Spiking neural networks (SNNs) are designed to be energy-efficient, but many neurons that do not contribute much still use valuable hardware resources. The authors create SparseForge, a system that smartly removes redundant neurons while considering hardware costs. They introduce new neuron models, clever data handling, and hardware designs to make SNNs run faster and use less energy. Tests show SparseForge keeps accuracy while greatly improving speed and energy use on real FPGA devices.

What this means in practice

  • For edge device engineers: Build hardware-accelerated spiking neural networks that run faster and more energy efficiently by pruning inactive neurons based on hardware-friendly methods.
  • For embedded system developers: Deploy energy-saving neural network models on FPGA platforms using SparseForge’s microarchitecture and dataflow optimizations to maximize computation per watt.

Authors

Kainan Wang, Sanhe Fu, Aili Wang

Abstract

Spiking Neural Networks (SNNs) provide an event-driven paradigm for energy-efficient edge computing. However, operation-level sparsity in SNNs does not necessarily eliminate the hardware costs associated with redundant output neurons, including neuron-state maintenance, partial-sum (psum) accesses, and on-chip storage requirements. This paper presents SparseForge, a hardware-aware sparse SNN acceleration framework that jointly optimizes neuron pruning, sparse dataflow, and accelerator microarchitecture. At the algorithm level, we introduce a gated leaky integrate-and-fire (G-LIF) neuron to learn differentiable neuron masks with binary spike semantics. A dual regularization scheme is further developed to align pruning with both synaptic-operation reduction and psum buffer capacity reduction. At the dataflow level, we design a sparse Column-Wise Product (CWP) dataflow with a variant of incremental spike encoding to improve weight-access regularity and reduce sparse-index overhead under dynamic spike activity. At the architecture level, we develop a mask-aware sparse processing unit (SPU) that supports compressed spike decoding, sparse implicit im2col conversion, neuron-mask-gated psum updates, and decoupled sparse execution. Experimental results across multiple SNN backbones and datasets show that SparseForge retains 1.75%--77.99% of neurons and 0.16%--62.36% of synaptic operations, with a configurable accuracy--efficiency trade-off. The buffer-capacity-aware regularization reduces the required psum buffer capacity by up to 96.18\%. FPGA implementations on a Xilinx ZCU9EG achieve 296.94--619.49 GOPS/W across the evaluated workloads. Across matched network and dataset settings, SparseForge provides 1.24--4.17 higher FPS and 1.50--7.18 higher GOPS/W than representative FPGA-based SNN accelerators, while reducing EDP by 1.70--19.13, with benchmark-dependent accuracy--efficiency trade-offs.