Reconfigurable matrix engine speeds signal and AI processing
A Cross-Domain Matrix Engine for Signal Processing and Deep Neural Net Inference
Hardware ArchitectureArtificial Intelligence
Summary
Many computer calculations, like those for wireless signals and AI, rely on multiplying grids of numbers. These two tasks prefer different ways to do these multiplications—signal tasks want quick results on small grids, while AI wants fast, energy-efficient results on big grids. The authors designed one adaptable engine that switches between two styles depending on the job. This means it can run both types of tasks faster and more efficiently than before, without needing two separate machines.
What this means in practice
- •For embedded system engineers: Integrate a single reconfigurable matrix engine to efficiently handle both low-latency signal processing and high-throughput DNN inference workloads on diverse devices.
- •For ai hardware designers: Use the adaptable engine design to improve speed and energy efficiency of neural network inference across varying batch sizes without major memory trade-offs.
Authors
Mehdi Safarpour
Abstract
Matrix multiplication dominates both PHY signal processing and Deep Neural Network (DNN) inference, yet these domains place opposing demands on the matrix engine. Signal processing requires low-latency execution of relatively small matrix operations, whereas DNN inference, particularly at large batch sizes, prioritizes high throughput and energy efficiency for large matrix multiplications. While offering efficient memory access profile, a Systolic Array (SA) adds fill and drain delays per matrix. Those can be amortized for large matrices but not for small matrices. Conversely, a tensor unit completes the kernel in few cycles but would require repeated reads of same operand tiles over large matrices. This makes SA too slow for signal processing and tensor too inefficient for AI. We show that this need not be a choice between systolic and tensor structuring of Multiply-Accumulate (MAC) units. We propose a single engine that operates primarily as a SA and, through low-cost reconfiguration, transforms into a tensor-style engine on a per-operation basis. MUXes create both the spatial reduction adder tree and temporal reduction accumulators from the existing MACs and registers within the systolic array. The proposed reconfigurable SA simulated using modified SCALESim simulator, verified on an Zynq 702 FPGA and synthesized using ASAP 7nm FinFET library. SCALE-Sim simulations shows up to 7.5x lower latency for small complex value PHY kernels. Even on DNN workloads, it achieves up to 1.65x speedups over vanilla SA execution with trivial increase of memory accesses, retaining the systolic array's memory access advantage. Synthesis results estimate reconfigurable fabric costs only 8% power and 13.6% area overheads.