FPGA accelerator speeds up sparse matrix calculations with efficient memory use
SUSpMV: A High Frequency Sparse Matrix Vector Multiplier on HBM Enabled FPGA written in SUS
Hardware ArchitectureDistributed, Parallel, and Cluster Computing
Summary
Sparse matrix-vector multiplication is a common math problem in computing that can be slow if not done efficiently. The authors created SUSpMV, a specialized hardware design on an FPGA that uses a new programming method to speed up these calculations. It organizes data and tasks cleverly to use all the memory channels and calculation units at high speed. This design is faster than previous work on the same chip, reaching close to the maximum possible speed.
What this means in practice
- •For hardware design engineers: Build FPGA accelerators for large sparse matrix computations used in scientific and machine learning applications with better speed and memory efficiency.
- •For high performance computing teams: Improve throughput of sparse linear algebra operations by deploying FPGA-based accelerators that utilize HBM and DDR memory effectively.
Authors
Lennart Van Hirtum, David Volz, Andreas Koch, Christian Plessl
Abstract
SUSpMV is a Sparse Matrix Vector multiplication (SpMV) accelerator written in the upcoming HDL SUS. By leveraging SUS's unique latency counting and inference mechanism, SUSpMV could be designed with very deep pipelines yet small design complexity overhead. This enables an efficient implementation which employs all 32 HBM channels on the Alveo U280 FPGA at 400MHz for streaming matrix data into 32 Compute Units (CUs). Input and output vectors are stored in DDR memory, which allows zero-overhead chaining of multiplications and increases overall system memory bandwidth by not sharing HBM bandwidth with the CUs. A CU processes the SpMV in tiles of width 1024 and a dynamically chosen height, up to 32768. Each CU is capable of accumulating the multiplications with up to 6 separate matrix entries per cycle, resulting in the combined theoretical peak computational throughput of 153.6 GFLOPs. The matrix storage format is designed to exploit density variation within a given matrix by dynamically switching between one representation optimized for denser regions, and a second optimized for sparser regions, allowing jumps of up to 255 rows between each entry. Evaluation demonstrates a 79% geometric mean improvement over prior work on the same platform. We achieve a peak throughput of 144.9 GFLOPs, or 94% of our theoretical computational throughput, compared to 98 GFLOPs reached by prior work.