Efficient mapping improves deep neural network hardware performance
IMPACT: Constraint-Aware Search for Imperfectly Factorized Accelerator Mappings
Hardware Architecture
Summary
Running deep neural networks on special hardware is tricky because arranging the work to fit the machine well is very complex. Most ways to organize the work either break hardware rules or waste resources. The authors designed a system called IMPACT that searches smarter by fixing bad plans instead of throwing them away, finding better ways to run these networks. This makes running the neural networks faster and more energy efficient across different types of hardware.
What this means in practice
- •For hardware accelerator engineers: Automatically generate efficient computation mappings that meet hardware constraints to optimize performance and energy use on flexible DNN accelerators.
- •For chip designers for novel computing architectures: Find feasible and near-optimal operation mappings on highly constrained accelerators such as compute-in-memory designs to improve runtime and energy efficiency.
Authors
Konstantinos Stavrakakis, Ilgaz Er, Theofilos Spyrou, Dimitrios Stavrakakis, Stephan Wong, Georgi Gaydadjiev, Said Hamdioui
Abstract
The efficiency of Deep Neural Network (DNN) accelerators depends on how each tensor operation is mapped onto the hardware. A mapping specifies how the computation is tiled across the memory hierarchy, ordered, and parallelized across the processing elements. These choices form a combinatorially large search space, where efficient mappings are hard to find. Most violate hardware constraints, and most remaining mappings underutilize the hardware. Existing mappers either search only part of this space to keep the search tractable, excluding the most efficient mappings, or sample the full space randomly, so most mappings they evaluate are infeasible. We present IMPACT, a mapper that searches the full space. IMPACT pairs a genetic algorithm with a constraint-aware repair operator that converts each infeasible mapping into a feasible one instead of discarding it. On a flexible accelerator, IMPACT outperforms state-of-the-art mappers in EDP, latency, and energy on every DeepBench workload. On a compute-in-memory (CIM) accelerator, whose constraints leave few mappings feasible, it reduces EDP by 66%, latency by 47%, and energy by 18% over the strongest prior mapper across five networks, and rivals hand-crafted expert mappings. Across flexible and highly constrained accelerators, IMPACT finds efficient mappings automatically. IMPACT will be released as open source.