CMM-MP device boosts memory capacity and speed for GPUs and CPUs
CMM-MP: A Novel Memory Pooling Architecture using Purpose-built Memory Device with CXL and Ethernet
Hardware Architecture
Summary
Some computer applications need more memory than usual, but adding more memory can be tricky. The researchers introduce CMM-MP, a special device that pools memory from many places using both fast connectors and common network technology. This device lets computers use hundreds of terabytes of extra memory as if it were one big memory block. It works without changing existing CPUs or GPUs and is faster and more efficient than other methods. The authors tested it with real software to show it works well.
What this means in practice
- •For data center engineers: Expand GPU and CPU memory capacity flexibly without hardware changes using CMM-MP’s memory pooling architecture.
- •For network architects: Implement large-scale memory pools combining CXL and Ethernet to enhance throughput and reduce latency beyond RDMA solutions.
Authors
Michael Wasef, Gongjin Sun, Edmund Au, Ranjitha Gopalakrishna, Elham Movahedian, Dongyang Li, Senthil Murugesapandian, Caroline Kahn, Shuyu Lyu, Brian Luu, Jason Scott, Namita Kumar, Mehdi Nik, Kwan Huen, Xuebin Yao, Amir Beygi, Luis Vitorio Cargnini, Ramdas Kachare
Abstract
CXL Memory Module - Memory Pooling (CMM-MP) introduces a novel memory pooling architecture to meet the unique requirements of emerging memory-intensive workloads. This industry-first, purpose-built memory-pooling device combines the benefits of Compute Express Link (CXL) with the scalability and cost advantages of Ethernet. Multiple CMM-MP devices together provide a flexible, composable, and disaggregated memory expansion solution. The CMM-MP enables 100s of TB capacity for GPU/CPU applications without modifications. The memory pool is exposed as a large-capacity logical aggregate memory, with all pool discovery, composition, and backend network datapath tasks fully offloaded to the device. CMM-MP supports both memory pooling and sharing usage modes. It provides an application latency of 2.7 us and up to 23 GB/s of memory throughput. Our evaluations demonstrate that CMM-MP significantly outperforms RDMA-based solutions in latency and throughput. We also evaluated real-world use cases of Valkey+YCSB and DiskANN using CMM-MP.