Spiking neural network system cuts audio denoising power use drastically
Zephyr: An Efficient Audio Denoising System Using Spiking Neural Networks Enabled With A Sparsity-Aware Flexible FPGA PE Array
Hardware Architecture
Summary
Audio denoising on small devices like wireless headphones uses a lot of battery power. The authors show that spiking neural networks, which work differently from usual neural networks, can reduce this power use by about 28 times when made hardware friendly. They designed a special digital circuit that takes advantage of this and demonstrated it working in real time on an FPGA chip. This means devices could have cleaner sound without draining batteries quickly.
What this means in practice
- •For mobile device engineers: Implement energy-efficient audio noise reduction in smartphones or wireless earbuds using hardware-friendly spiking neural networks.
- •For embedded hardware designers: Design flexible FPGA-based accelerators optimized for sparse spiking neural network workloads in audio processing systems.
Authors
Cheng-En Chang, Chi-Wei Kao, Chung-Lun Yang, Yan-Lin Jiang, Yi-Chen Huang, Sebastian Fieldhouse, Kea-Tiong Tang
Abstract
In this work we look to neuromorphic computing to solve the power consumption problem that audio denoising neural networks face on edge devices like smartphones, wireless headphones and hearing aids. Spiking neural networks (SNNs) have the potential to solve this problem due to their high activation sparsity and low complexity, however many SOTA SNNs require hardware that supports a mixture of operations to be able to fully perform inference. To solve this problem, we convert SOTA audio denoising neural network Spiking-FullSubNet to a hardware friendly version showing that via QAT and activation function simplification we can achieve $\approx28\times$ improvement in power consumption to 52.9nJ per 32ms audio frame when calculated for custom digital hardware in a 45nm process node. We then propose a digital circuit which by means of a sparsity-aware flexible PE array can perform inference of the heterogeneous compute load of Spiking-FullSubNet, and validate this circuit on a PYNQ-Z1 FPGA achieving a real-time factor of 0.727 at 100MHz.