NeuroFlex boosts deep learning speed and energy use by mixing modes
NeuroFlex: Lossless Element-Level ANN-SNN Co-Execution for Efficient Sparse Inference
Hardware ArchitectureMachine Learning
Summary
Often, specialized chips run only one type of deep learning model, either artificial neural networks (ANNs) or spiking neural networks (SNNs), leading to inefficiencies when both are useful. The authors introduce NeuroFlex, a chip that can switch at the level of individual outputs between ANN and SNN modes without losing accuracy. This fine-grained switching improves how busy the chip's processing units stay, cutting delays and energy use significantly. Their smart scheduler decides which mode to use for each output, making inference faster and more energy efficient across various tasks.
What this means in practice
- •For chip designers: Integrate NeuroFlex methods to create efficient accelerators that handle mixed ANN and SNN workloads with better energy and speed profiles.
- •For mobile device developers: Design energy-efficient devices that run deep learning tasks faster by using element-level adaptive mode switching between ANN and SNN processing.$Commercial implications: Enables production of mobile AI chips with improved battery life and responsiveness by combining ANN and SNN execution.
Authors
Varun Manjunath, Pranav Ramesh, Gopalakrishnan Srinivasan
Abstract
Sparse DNN accelerators specialize in ANN or SNN execution, leaving energy or latency on the table when workload characteristics vary within a layer. Hybrid accelerator designs that switch modes at layer or tile granularity suffer from low PE utilization since one core type idles whenever the other is active. NeuroFlex is the first accelerator to assign every output element independently to ANN or SNN execution mode with zero accuracy loss. We extend integer-exact ANN-SNN equivalence from layers to individual output elements, thereby enabling mode switching with no conversion error. An offline cost-guided scheduler scores each element by its marginal energy-delay trade-off and packs work across PEs, achieving 97-99% PE utilization compared to 40-45% for layer-wise hybrids. NeuroFlex reduces EDP by 57-67% over a strong ANN-only baseline and delivers up to 2.5x speedup over a dual-sparse SNN-only baseline. Our cost-guided scheduler improves throughput by 16-19% over random element assignment across vision, language, and transformer workloads.