Simple model helps predict smartnic performance bottlenecks early
Towards Simple Models of Complex SmartNICs
Distributed, Parallel, and Cluster ComputingNetworking and Internet Architecture
Summary
SmartNICs are specialized network cards used by cloud providers to handle complex tasks like encryption and traffic monitoring without burdening the main server. These devices have many parts and can be difficult to program and understand. The authors present a model called ZRAM that maps programs onto SmartNIC hardware to predict performance limits before writing code. They tested this approach using applications like detecting network attacks and decision tree processing, and identified design patterns to improve programming. This model applies to various SmartNICs and may guide future software and hardware development.
What this means in practice
- •For cloud infrastructure engineers: Predict performance bottlenecks in SmartNIC applications early to choose efficient hardware and software designs without extensive trial and error.
- •For network security teams: Improve DDoS detection performance on SmartNICs by following proven design patterns and placement strategies from the ZRAM model.
Authors
Robert Chang, Teng Jiang, Wonsup Yoon, Daehyeok Kim, Sam Kumar, George Varghese
Abstract
Cloud vendors push ambitious in-network processing (e.g., crypto, telemetry) onto the NIC to offload servers even as link rates climb to terabit speeds. Vendors have responded with heterogeneous SmartNICs. For example, NVIDIA BlueField-3 interposes---between the wire and the host CPUs---a line-rate eSwitch, a multithreaded Data-Path Accelerator, general-purpose ARM cores, and a sea of fixed-function accelerators. These devices are notoriously hard to program, and harder still to predict. Applications can be implemented in many ways, with each choice potentially hitting a different bottleneck. A designer ideally needs to know---cheaply, and before a line of code is written---feasible choices and their bottlenecks, and design patterns to improve performance. Our paper offers a starting point to answer these questions using what we call the ZRAM model. It pairs a platform graph of processing zones and their channels with a program graph of tasks and their traffic fractions. The application designer or a compiler chooses a placement that maps the program graph onto the platform graph. Three metrics computed directly from this mapping---capability, roofline, and capacity---score the placement, deciding its feasibility and naming the bottleneck resource. We use a DDoS detector as a primary case study, and briefly explore two other applications, decision-tree inference and RDMA traversal. We distill seven design patterns for programming SmartNICs including a key one we call sifting. ZRAM generalizes to other SmartNICs such as Intel IPU E2200 and AMD Pensando Salina 400, and opens a new research agenda that includes compilers and hardware design.