Benchmark suite evaluates communication compression in HPC and AI workloads
Tools-CC-Bench: a Benchmark Suite for Collective Communication with Compression in HPC and AI Workloads
Distributed, Parallel, and Cluster Computing
Summary
Moving large amounts of data quickly is a challenge in high-performance computing (HPC) and large AI models. Compressing communication can help, but it's hard to measure how well this works with current tests. The authors created CC-Bench, a suite that mimics real-world conditions to test and compare different compression methods during communication. It looks at speed, resource use, and accuracy using real datasets, helping people find the best choices for their computing tasks.
What this means in practice
- •For hpc system administrators: Assess communication compression strategies to improve scalability and resource use in supercomputer clusters.
- •For ai infrastructure engineers: Evaluate compression methods to reduce overhead and maintain accuracy when training large language models across GPUs.
Authors
Haozhe Fan, Wei Wang, Xingchen Liu, Man Liu, Xingjian Tian, Haoquan Long, Zedong Liu, Daran Sun, Jinwu Yang, Bo Yang, Jie Liu, Yonggang Che, Hairui Zhao, Guangming Tan, Dingwen Tao
Abstract
Distributed HPC and LLM workloads increasingly require efficient communication for scalability, yet growing data movement has become a major performance bottleneck. Communication compression can reduce this overhead and complement execution-level optimizations, but its benefits remain difficult to assess because existing benchmarks lack support for diverse backends, realistic datasets, application-specific accuracy metrics, and overlap-induced resource contention. We present CC-Bench, a lightweight, extensible, and application-oriented benchmark suite for evaluating communication compression under realistic execution conditions. CC-Bench uses declarative application-environment modeling to decouple profiling logic from communication libraries, datasets, and fidelity metrics, enabling portable cross-library evaluation. It further combines function-level interception and hardware counter monitoring to characterize per-phase latency, hardware utilization, numerical fidelity, and computation interference. With representative datasets from HPC and LLM workloads, CC-Bench evaluates three compression-enabled communication libraries on CPU and GPU clusters, revealing accuracy-performance trade-offs and bottlenecks to guide practical deployment and optimization.