Event-Driven Language Models with Sparse Neural Activity for Neuromorphic Hardware
2026-08-31 • Neural and Evolutionary Computing
Neural and Evolutionary ComputingMachine Learning
AI summaryⓘ
The authors address the high memory and computation needs of large language models by using state-space models with linear attention, but these still have costly dense operations. They create a way to make these models sparse by ignoring small activation values while keeping important ones, which cuts down the calculations significantly without hurting performance. Their method works especially well on special multi-core hardware that efficiently handles sparse and event-driven tasks, potentially making it much faster and more energy-efficient than typical GPUs. This suggests their approach could make running advanced language models on edge devices more practical.
Large Language ModelsTransformerState-Space ModelsLinear AttentionSparse Neural NetworksQuantizationKV CacheNeuromorphic ComputingEvent-Driven ExecutionMulti-Core Processors
Authors
Simon Richter, Ruhai Lin, Jason Yik, Taylor Kergan, Rui-Jie Zhu, Farshad Moradi, Jason Eshraghian
Abstract
Inference with transformer-based large language models (LLMs) is often limited by the memory-bound KV cache and quadratic attention cost. State-space models (SSMs) mitigate this through linear attention and fixed-size recurrent states, but their large dense linear projections remain computationally expensive even after quantization. We introduce a method that induces sparse neural activity in heavily quantized linear-attention models with minimal performance loss. Activations below a per-projection trainable threshold ($\pm Δ$) are nullified while preserving crucial outliers, achieving comparable performance to dense models with up to 4$\times$ fewer effective arithmetic operations. Targeting a multi-core, multi-chip neuromorphic platform, where event-driven execution converts unstructured sparsity into throughput at both the compute and communication levels, a capability GPU architectures fundamentally lack, we project up to 37$\times$ higher throughput and 16$\times$ lower power versus edge GPU inference of a comparable transformer-based model, and up to 5.4$\times$ improvements over the non-sparsified baseline. These results position sparse, quantized linear-attention models as a natural fit for deploying LLMs on event-driven multi-core platforms.