FlashGPU simulates modern GPUs for AI to analyze bottlenecks
FlashGPU-sim: Enabling GPU Modeling for Modern Architectures and AI Workloads
Hardware ArchitectureDistributed, Parallel, and Cluster Computing
Summary
Modern AI uses special graphics cards called GPUs that have complex features to speed up calculations. Older simulation tools can't mimic these new features or run the latest AI programs accurately. The authors built FlashGPU-sim, a detailed simulation tool that copies how today’s GPUs work, letting people test and improve AI hardware designs using real AI workloads. This tool runs faster with multiple threads and matches real GPUs closely, helping engineers find and fix performance problems.
What this means in practice
- •For gpu architects: Use FlashGPU-sim to evaluate design trade-offs and performance bottlenecks in AI-focused GPU hardware before building silicon.
- •For ai compiler engineers: Run real AI operators generated by modern compilers like Triton on FlashGPU-sim to test and optimize code for new GPU features.
Authors
Siying Yu, Yixun Hong, Guozhi Qiu, Jingci Liu, Feng Gu, Chenbo Geng, Zhengrong Wang, Chen Zhang, Bei Yu
Abstract
As AI becomes increasingly ubiquitous, modern AI systems are shaped by a tight software-hardware co-design loop. Later GPUs expose features such as asynchronous data movement, tensor core pipelines, and fine-grained synchronization that high-performance kernels aggressively exploit, while emerging application behaviors increasingly influence the next generation of hardware design. Unfortunately, the latest open-source simulators for NVIDIA GPUs focus on architectures and software stacks from roughly six years ago. Therefore, they cannot support many state-of-the-art AI kernels generated by modern compiler stacks, e.g. Triton, or accurately model the hardware features they depend on. As a result, architects lack a credible platform for analyzing bottlenecks in this flywheel or evaluating design trade-offs for future AI systems. To bridge this gap, we present FlashGPU-sim, an open-source, execution-driven, cycle-accurate GPU simulator for modern AI workloads. FlashGPU-sim faithfully models modern hardware features such as asynchronous data movement, fine-grained synchronization, tensor-core execution, and distributed shared memory. A Triton extraction front-end allows direct simulation of optimized AI operators without manual porting, while multi-threaded execution makes large-scale software-hardware co-design practical. Across 131 workload configurations on RTX 5090, H100, and B200, FlashGPU-sim achieves a cycle-level MAPE of 5.24%, while multi-threaded simulation reaches a 7.86x speedup with 16 host threads. An H100 case study further demonstrates its utility for microarchitectural design exploration.