Bit-sparsity aware hardware speeds up generic matrix multiplication

Bit-Sparsity Aware Acceleration with Compact CSD Code on Generic Matrix Multiplication

Hardware Architecture

Summary

Matrix multiplication is central to many computer tasks, especially in AI, but it can be slow and energy-hungry. The authors designed a new hardware architecture called Bit-Cigma that uses a special number encoding and processing techniques to speed up calculations for both floating-point and integer data. Their design reduces the computing work by focusing on the important bits, leading to faster and more energy-efficient operations. This improvement works on general computing tasks beyond just neural networks.

What this means in practice

  • For chip designers: Develop hardware accelerators that speed up matrix multiplication with lower energy and area costs using bit-sparsity aware encoding for both floating-point and integer data.$Commercial implications: Enables new accelerator chips with better performance and power efficiency for AI and generic computing workloads.
  • For high-performance computing teams: Integrate bit-sparsity aware processing units like Bit-Cigma into systems to boost matrix computations in scientific and engineering simulations.

Authors

Zixuan Zhu, Xiaolong Zhou, Chundong Wang, Li Tian, Zunkai Huang, Yongxin Zhu

Abstract

The ever-increasing demand for matrix multiplication in artificial intelligence (AI) and generic computing emphasizes the necessity of efficient computing power accommodating both floating-point (FP) and quantized integer (QINT). While state-of-the-art bit-sparsity-aware acceleration techniques have demonstrated impressive performance and efficiency in neural networks through software-driven methods such as pruning and quantization, these approaches are not always feasible in typical generic computing scenarios. In this paper, we propose Bit-Cigma, a hardware-centric architecture that leverages bit-sparsity to accelerate generic matrix multiplication. Bit-Cigma features (1) CCSD encoding, an optimized on-chip sparsification technique based on canonical signed digit (CSD) representation; (2) segmented dot product, a multi-stage exponent matching technique for long FP vectors; and (3) the versatility to efficiently process both FP and QINT data types. CCSD encoding halves the cost of CSD encoding while achieving optimal bit-sparsity, and segmented dot product improves both accuracy and throughput. Bit-Cigma cores are implemented using 65 nm technology at 1 GHz, demonstrating substantial gains in performance and efficiency for both FP and QINT configurations. Compared to state-of-the-art Bitlet, Bit-Cigma achieves 3.2\texttimes{} performance, 6.1\texttimes{} area efficiency, and 15.3\texttimes{} energy efficiency when processing FP32 data while ensuring zero computing error.