Toki profiles memory performance on FPGA systems with many RISC-V cores
Toki: Profiling HBM Performance on FPGA Systems with RISC-V Soft Cores and PCIe Host DMA Traffic
Hardware Architecture
Summary
High-bandwidth memory (HBM) on FPGA accelerator cards plays a big role in speeding up data center tasks. The authors created Toki, a system that measures how this memory performs when running many RISC-V soft processor cores on an FPGA while also handling data from a host computer over PCIe. This helps reveal how different factors like core arrangement, memory access methods, and traffic from the host affect speed and efficiency. Their tests used a real AMD FPGA card and showed how real-world applications stress the memory differently than simple test patterns.
What this means in practice
- •For fpga system designers: Profile and optimize high-bandwidth memory use in FPGA accelerators running multiple RISC-V cores with real application and host traffic scenarios.
- •For data center hardware teams: Evaluate how host-to-FPGA DMA traffic impacts memory performance on accelerator cards to improve system throughput and latency.
Authors
Andrea Galimberti, Andrea Motta, Gianni Antichi, Davide Zoni
Abstract
Programmable RISC-V soft cores are becoming more widespread in data-center scenarios, making it crucial to design efficient systems that deploy them on FPGA chips with HBM memory. Toki, released as open source, is the first hardware-software framework that enables profiling the performance of HBM on FPGA accelerator cards by jointly considering (i) the execution of workloads on RISC-V soft cores instantiated on the FPGA and (ii) the injection of memory traffic from the host system via DMA over PCIe, providing insights that cannot be obtained with synthetic traffic generators alone. Extensive experiments target an AMD Alveo U55C card, deploying up to 60 RISC-V compute cores and stressing its HBM2 memory through real-world applications and microbenchmarks with user-defined access patterns. Results showcase how Toki can effectively profile the impact on HBM performance of the compute cores' organization, the workload's memory access patterns, data locality, contention over the memory controllers, and host traffic.