Co-Fabric improves communication between multiple xPUs across hosts
Co-Fabric: Breaking Host-Domain Boundaries for Unified xPU Interconnection
Distributed, Parallel, and Cluster Computing
Summary
Big AI models often need more computing power than a single processor (xPU) can provide, so they use many xPUs across different computer systems, but making them talk to each other quickly is hard. The authors present Co-Fabric, a new way to connect these xPUs across different host computers with a simple and fast communication method. Co-Fabric reduces delays and increases data speed, making AI tasks run faster and more efficiently while using less power and cost. This technology could improve how AI centers process large models by streamlining communication between many processors.
What this means in practice
- •For data center engineers: Build AI hardware with more efficient multi-xPU connections to speed up large model inference and reduce communication overhead.
- •For network hardware designers: Design cross-host interconnect devices that support unified addressing and ultra-low latency for large-scale AI systems.
Authors
Zhen Peng, Jiaming Huang, Chaofan Chen, Zhao Zhang, An Wu, Baoyang Liu, Xinglong Wang, Tanlong Ci, Jinfeng Li, Xueke Duan, Hao Wang, Xi Chen, Shunshun Zhang, Zhiyuan Su, Zhu Cao, Zhichong Dou, Shaohua Wu, Lu Jing, Yue Yuan
Abstract
Large-model parameters have grown beyond the capacity of a single xPU, dispersing across multiple xPUs spanning distinct host domains, where xPU-to-xPU communication dominates overall system efficiency. Existing scale-up interconnect remains inadequate: network-based solutions built on Ethernet--such as RoCE (RDMA over Converged Ethernet)--introduce specific message-semantics and protocol-stack characteristics, and rely on fragmented per-host addressing, while conventional host-based fabrics are confined to a single host domain and lack cross-host unified addressing. This paper presents Co-Fabric, a bus-based interconnect that, unlike conventional bus designs, breaks host-domain boundaries to deliver unified xPU interconnection for scale-up superpods. Co-Fabric makes three contributions: a streamlined four-layer protocol stack achieving nanosecond-scale processing latency with native reliability; a cross-domain scaling and P2P mechanism that routes using port identifiers embedded in the packet header; and a unified address space built on shadow-device auto-enumeration. On a 64-xPU 3D-Mesh system, Co-Fabric cuts inter-node communication latency by over 50% and improves bandwidth by 2-5x over RoCE, accelerating DeepSeek R1 inference by 30%-80%. Moreover, since its streamlined four-layer protocol stack and higher data-communication efficiency reduce protocol and processing overhead relative to the Ethernet-based RoCE stack, Co-Fabric cuts the cost and power of the interconnect itself by up to 80% and 5%, respectively. These results demonstrate Co-Fabric's advantage for AI computing centers.