Memory-bound CFD runs faster and cheaper with smart AMD EPYC settings
Data-Driven Prediction of Optimal MPI Rank Placement and Node Count for Memory-Bound CFD on AMD EPYC Clusters
PerformanceMachine Learning
Summary
Running computer simulations for fluid flow on AMD EPYC clusters needs decisions about how many nodes and processor cores to use. The authors studied hundreds of tests to find that for small problems, using one node is fastest and cheapest, while big problems benefit most from many nodes, being both faster and cheaper. Medium-sized problems show a real trade-off between speed and cost. They also found a simple way to predict nearly optimal settings without complex models. This can help users save time and money when running these simulations.
What this means in practice
- •For high performance computing teams: Optimize node and processor core use for memory-bound CFD simulations on AMD EPYC clusters to balance speed and cost effectively.
- •For cloud infrastructure managers: Predict near-optimal resource configurations for fluid dynamics jobs to reduce compute time and node hours on AMD EPYC cloud clusters.
Authors
Marcin Lawenda, Łukasz Szustak, Aleksandra Krasicka
Abstract
Users who run a memory-bound OpenFOAM case on an AMD EPYC cluster must decide how many nodes to use and how many MPI ranks to place on each core complex. The fastest choice is often not the cheapest one. This paper revisits our earlier analytical model of this performance and energy trade-off with 560 measured configurations of the motorBike case on five Zen 4 and Zen 5 EPYC processors, including socket power from processor telemetry. On up to four nodes, small meshes are both fastest and cheapest on a single node on most processors. For the largest mesh the largest node count is both the fastest and the cheapest option, because its speedup is superlinear. A genuine trade-off arises only for medium meshes, where full speed costs a median of 20 to 97% more node hours than the cheapest option. The L3 miss rate and the halo traffic per cell are consistent with this pattern. For a new mesh of the same case, the configuration that was optimal on the nearest measured mesh is near optimal in 75 to 85% of the decisions. Running the two configurations it recommends for speed and for cost raises this share to 85 to 90% for runtime and node hours and determines the trade-off with median errors of 3 to 5 percentage points. Neither gradient boosted trees trained on the same data nor the earlier polynomial model improves on this simple procedure.