RapID boosts memory computing by processing whole rows fast
RAPID: Row-Parallel Arithmetic Processing in DRAM
Hardware Architecture
Summary
Moving data between a computer's processor and memory can slow down tasks a lot. The paper presents RAPID, a way to do math calculations right inside the computer memory but working with data arranged just like the processor expects it. This avoids slow changes in how the data is arranged and lets the memory work on many tasks at once, speeding things up. The authors tested RAPID on typical machine learning tasks and found it to be nearly six times faster than a previous method.
What this means in practice
- •For hardware designers: Build memory chips that perform arithmetic directly on row-parallel data layouts to speed up machine learning workloads without costly data rearrangement.
- •For ml system engineers: Improve performance of machine learning inference by integrating RAPID to reduce memory bottlenecks during computation-heavy tasks.
Authors
William C. Tegge, João Paulo Cardoso de Lima, Shouzhi Fang, Jeronimo Castrillon, Alex K. Jones
Abstract
Processing-using-memory (PUM) architectures perform computation directly within DRAM to reduce costly data movement between memory and processors. Because charge-sharing operations are confined to individual bitlines, existing DRAM-PUM architectures reorganize data into column-oriented, bit-serial representations. This organization is fundamentally incompatible with the row-oriented, word-parallel layouts used by conventional processors and accelerators, requiring expensive data-layout transformations whenever computation transitions between PUM and conventional execution. In this paper, we present RAPID, a Row-parallel Arithmetic Processing-In-DRAM architecture. RAPID augments the DRAM subarray with two lightweight extensions: migration cells that enable localized horizontal data movement between neighboring bitlines and inversion cells that provide efficient in-array logical inversion. These primitives enable RAPID to operate directly on row-parallel, bit-parallel data, preserving CPU-compatible layouts while exploiting the massive parallelism of the DRAM subarray. In particular, RAPID demonstrates that localized horizontal communication is sufficient to realize shallow arithmetic networks and efficient parallel reduction for multiplication, reducing arithmetic latency while preserving throughput and eliminating costly data-layout transformations, all while maintaining the conventional DRAM array organization. We demonstrate the feasibility and overhead of augmenting DRAM subarrays with migration and inversion cells through detailed transistor-level layout and SPICE-validated circuit simulations. Using the RAPID compiler it is possible to evaluate the performance and data reorganization tradeoffs to ensure the best execution across combined CPU and PUM. Evaluating RAPID on 19 MLPerf benchmarks, there is a 5.9x higher end-to-end performance compared to SIMDRAM for DDR4 PUM execution.