Mixed ADC resolution improves energy efficiency and accuracy in analog AI computing

SMART-AIMC: Sensitivity-Aware Mixed ADC Resolution with Fine-Grained Tunability for Analog In-Memory Computing

Hardware ArchitectureArtificial Intelligence

Summary

Analog in-memory computing (IMC) speeds up AI tasks by doing math directly inside memory, but it can lose accuracy due to noisy signal conversions. The authors developed SMART-AIMC, a way to smartly pick different precision levels for these conversions based on how sensitive each part of the calculation is, improving accuracy and saving energy. They also reorder computations to better use hardware capabilities without slowing down. Their method achieves much better energy efficiency than standard digital AI systems while keeping accuracy very close.

What this means in practice

  • For hardware designers: Design energy-efficient mixed-resolution ADC systems that optimize accuracy in analog AI accelerators for GEMM computations.
  • For cloud ai infrastructure teams: Implement SMART-AIMC methods in AI accelerator platforms to reduce power consumption without significant accuracy loss during large-scale model inference.

Authors

Yung-Chin Chen, Ourania Stouraiti, Naveen Verma

Abstract

Analog In-Memory Computing (IMC) is a promising paradigm for accelerating GEMM-based AI workloads by executing computations directly within the memory array in the analog domain. While IMC delivers substantial energy and throughput improvements, it can suffer from significant task accuracy degradation due to analog-to-digital converter (ADC) quantization noise, which occurs at the GEMM output and is thus different from weight/activation quantization, which occurs at the GEMM input. Mixed ADC resolution allows IMC modules to choose between high- and low-resolution ADC modes with different energy costs, offering a compelling accuracy-energy trade-off. However, selecting ADC resolutions across IMC units while respecting hardware granularity constraints remains challenging. In this paper, we propose SMART-AIMC, a Sensitivity-aware Mixed ADC Resolution framework with fine-grained Tunability for Analog In-Memory Computing. SMART-AIMC correlates the algorithmic task loss to hardware ADC quantization noise in the form of an ADC Channel Sensitivity (ACS) metric. Using this metric, SMART-AIMC systematically, and in a disciplined manner, allocates ADC resources to critical computations. Furthermore, we introduce a sensitivity-based channel reordering scheme that overcomes the hardware granularity constraints with no runtime overhead and approaches the optimal ADC quantization possible for the model. Evaluated across diverse workloads, SMART-AIMC unlocks a superior accuracy-energy Pareto frontier, providing (1) higher task accuracy at matched energy budgets and (2) fine-grained tuning across efficiency targets, from aggressive energy savings to near-digital inference accuracy. At the latter operating point, SMART-AIMC achieves $6.99\times$ higher modeled system-level energy efficiency for linear projections than the INT8 digital baseline with negligible accuracy degradation.