Deep learning compilers optimize complex models on varied hardware
Deep Learning Compilers: Architecture, Optimization, and Empirical Characterization
Machine LearningHardware Architecture
Summary
Deep learning models are becoming more complex and run on many different computer types, which makes it tricky to get them to run efficiently. The authors study special software called deep learning compilers that transform these models so they run faster and use less power on different devices. They break down how these compilers work and test how well their optimizations perform on real hardware. Their experiments show that the benefits depend a lot on the kind of model and that some improvements can take many runs to pay off. This work helps engineers understand how to better design and use deep learning compilers.
What this means in practice
- •For machine learning engineers: Improve the speed and efficiency of deploying complex DL models on varied hardware using compiler optimizations tuned to workload types.
- •For graphics hardware developers: Design hardware features and APIs that better support DL compiler optimizations, especially fusion and dynamic shapes.
Authors
Jinghao Wen, Ruixuan Wang, Xun Jiao
Abstract
As deep learning (DL) models become increasingly complex and hardware architectures grow more heterogeneous, DL compilers play a critical role in optimizing computational efficiency and enabling scalable deployment. This paper presents a systematic analysis of DL compilers, covering their architectural evolution, compilation workflows, optimization techniques, and emerging research directions. We organize DL compiler architectures into frontend, middle-end, and backend stages, emphasizing the role of the middle-end in connecting high-level model representations with hardware-specific optimizations. We further examine representative DL compilers and recent optimization techniques across different compilation stages. Beyond architectural and technical analysis, we conduct an empirical evaluation of compiler behaviors to investigate the effectiveness of graph optimization, compilation overhead, and dynamic-shape handling. Experiments using PyTorch/TorchInductor on an NVIDIA RTX A6000 demonstrate that compiler benefits depend strongly on workload characteristics, with speedups of up to 8.21\times for fusion-friendly computation graphs, while other workloads exhibit limited improvements. Our results further reveal that compilation overhead can require thousands of executions to amortize, and dynamic-shape compilation reduces cumulative shape-transition costs by approximately 70\% at the expense of modest steady-state performance degradation. By combining systematic architectural analysis with quantitative experimental evidence, this work provides insights into the design, optimization, and practical deployment of modern DL compilers.