Mamba state-space model kernels fit well on programmable CGLA chips
Mamba-Family State-Space Model Kernels on a Programmable CGLA
Hardware Architecture
Summary
Running AI models on small devices like phones or sensors is hard because they use a lot of power and data movement. The authors looked at a special type of AI model called Mamba that uses a simpler way to process sequences instead of the usual attention method. They tested how parts of this model run on a new kind of chip that can be programmed to do many small operations in parallel. Their findings show that some parts of the model fit very well on this chip design, while others have performance limits due to the chip’s architecture. This means certain AI model parts can run faster and more efficiently on these chips, but others need special handling.
What this means in practice
- •For embedded system engineers: Improve inference speed and power efficiency for state-space model AI on programmable linear array chips in edge devices.
- •For gpu compiler developers: Optimize kernel mapping strategies for linear recurrence models to reduce bottlenecks in sequence processing workloads.
Authors
Takuto Ando, Yasuhiko Nakashima
Abstract
Edge and embedded inference is constrained by power and data movement. Mamba-family state-space models replace attention with sequence-linear recurrence, but their inference path combines dense projections, short-reduction SSD kernels, and recurrent-state updates. This paper maps these kernel groups onto IMAX, a programmable CPU-Grounded Linear Array (CGLA), and measures them from kernel execution to token-level integration. Projection kernels match the long-reduction IMAX pipeline, whereas SSD Step-1 is limited by short reductions and kernel-boundary overheads. Mamba-130M token-level integration identifies projection GEMV as the decode bottleneck. These results show that programmable CGLAs fit long-reduction projection kernels, while SSD and decode-time projection support require boundary reduction and persistent-weight execution.