Controller design improves long-span error correction for high-bandwidth memory
REACH: Controller-Managed Long-Span ECC for HBM AI Inference
Hardware Architecture
Summary
High-bandwidth memory (HBM) can experience errors that affect AI inference performance, and stronger error correction is needed. The paper presents REACH, a controller design that combines small, common error corrections with a specialized long-span code for rare errors to improve efficiency. This design reduces hardware area and power usage while maintaining high data throughput. The authors tested REACH with simulations and found it better than previous direct long-span error correction methods.
What this means in practice
- •For hardware engineers: design memory controllers that effectively manage high error rates in HBM used for AI inference workloads
- •For chip architects: optimize area and power consumption in AI inference chips with HBM by integrating controller-managed long-span error correction
Authors
Rui Xie, Yunhua Fang, Asad Ul Haq, Linsen Ma, Sanchari Sen, Swagath Venkataramani, Liu Liu, Tong Zhang
Abstract
High-Bandwidth Memory (HBM) cost motivates stronger controller protection that can support a wider range of device error rates. Long-span error-correcting codes provide stronger protection at a comparable code rate, but a direct implementation couples small accesses to span-wide state and requires costly decoding at HBM bandwidth. Read-dominated LLM decode offers a favorable setting: sequential reads support span aggregation, while sparse writes limit parity-update traffic. This paper presents REACH, a controller microarchitecture that uses established inner codes to correct common errors and identify unresolved chunks, reserving a long outer code for known-erasure repair. Differential parity bounds write traffic, and a co-designed endpoint preserves 32\,B transactions without an extra data burst. Ramulator2 sustains 1.88\,TB/s of application traffic at the highest error stress, while separate full-interface sizing supports a 2.69\,TB/s application target using ASAP7-synthesized kernels. At this analytical target, REACH's nominal composition uses 55.8\% less controller area and 57.7\% less modeled power than the evaluated mean-work direct-long design, showing the benefit of reserving long-span recovery for exceptional requests.