Benchmark measures communication speed and accuracy in AI and HPC systems
Tools-CC-Bench: a Benchmark Suite for Collective Communication with Compression in HPC and AI Workloads
Distributed, Parallel, and Cluster Computing
Summary
Moving data between computers in large AI and high-performance computing (HPC) systems can slow things down. Compressing the data helps reduce this traffic, but it’s hard to know how well different methods work because existing tests are limited. The authors created a new benchmark called CC-Bench that tests compressed communication in realistic settings using real applications and datasets. This tool checks how fast and accurate the communication is, and whether other computing tasks get slowed down. Their tests help developers understand trade-offs and improve future communication tools.
High-performance computingCommunication compressionBenchmarkingDistributed systemsLatencyHardware utilizationNumerical fidelityLarge language modelsGPU clustersData movement
Authors
Haozhe Fan, Wei Wang, Xingchen Liu, Man Liu, Xingjian Tian, Haoquan Long, Zedong Liu, Daran Sun, Jinwu Yang, Bo Yang, Jie Liu, Yonggang Che, Hairui Zhao, Guangming Tan, Dingwen Tao
Abstract
Distributed HPC and LLM workloads increasingly require efficient communication for scalability, yet growing data movement has become a major performance bottleneck. Communication compression can reduce this overhead and complement execution-level optimizations, but its benefits remain difficult to assess because existing benchmarks lack support for diverse backends, realistic datasets, application-specific accuracy metrics, and overlap-induced resource contention. We present CC-Bench, a lightweight, extensible, and application-oriented benchmark suite for evaluating communication compression under realistic execution conditions. CC-Bench uses declarative application-environment modeling to decouple profiling logic from communication libraries, datasets, and fidelity metrics, enabling portable cross-library evaluation. It further combines function-level interception and hardware counter monitoring to characterize per-phase latency, hardware utilization, numerical fidelity, and computation interference. With representative datasets from HPC and LLM workloads, CC-Bench evaluates three compression-enabled communication libraries on CPU and GPU clusters, revealing accuracy-performance trade-offs and bottlenecks to guide practical deployment and optimization.