Combining GPUs and WSEs boosts AI tasks but has limits
When Does Heterogeneity Pay Off? Understanding GPU-WSE System Complementarity
Distributed, Parallel, and Cluster ComputingHardware Architecture
Summary
Different types of AI processors, like GPUs and WSEs, work in very different ways. Combining them can sometimes make AI calculations faster, but it also adds complexity that can slow things down. The paper shows when mixing these processors helps and when it creates bottlenecks. The authors develop techniques to measure this balance and a model to predict performance gains from using both together.
What this means in practice
- •For ai system designers: Optimize the design of AI hardware platforms by predicting when combining GPUs and WSEs improves performance versus when it causes slowdowns.
- •For cloud infrastructure teams: Plan and deploy heterogeneous AI accelerator clusters with a better understanding of workload-level trade-offs between GPUs and WSEs.
Authors
Ziyu Hu, Zhijing Ye, Jiamin Wang, Zhiqing Zhong, Zhendong Mi, Shaoyi Huang, Weijian Zheng, Rajkumar Kettimuthu, Xiaodong Yu
Abstract
As AI accelerator architectures become increasingly diverse, integrating different architectures into a unified system is emerging as a new heterogeneous computing paradigm. GPUs and WSEs differ substantially in compute, memory, and communication organization, creating both architectural complementarity and additional overheads, yet how to translate this heterogeneity into practical performance gains remains unclear. To address this gap, we propose a hierarchical benchmarking methodology to characterize GPU--WSE heterogeneous execution and the conditions for exploiting complementarity, together with an analytical performance model that identifies when heterogeneity becomes a bottleneck and when it yields system-level performance gains.