Multi-engine accelerators improve deep learning speed and energy use
MEDEM: Multi-Engine DL Accelerator Design Methodology
Hardware ArchitectureDistributed, Parallel, and Cluster ComputingPerformance
Summary
Deep learning tasks involve many different types of calculations, and a single type of computer chip often struggles to do them all efficiently. The authors developed a step-by-step method called MEDEM that helps design multi-engine chips, combining different kinds of engines tailored to handle varied tasks. By carefully choosing and designing these engines together, MEDEM finds chip designs that run deep learning workloads faster and with less energy. Their tests show these designs outperform current ones for many different deep learning models and hardware budgets.
What this means in practice
- •For hardware design teams: Design custom multi-engine accelerators that efficiently handle diverse deep learning workloads with optimized speed and energy consumption.
- •For data center operators: Deploy accelerators selected via MEDEM to improve throughput and energy efficiency across mixed deep learning applications.
Authors
Fareed Qararyah, Mohammad Ali Maleki, Pedro Trancoso
Abstract
Multi-engine deep learning (DL) accelerators are becoming increasingly prevalent as they address the heterogeneity and growing complexity of modern DL workloads. To efficiently process diverse DL workloads, these accelerators must incorporate combinations of engines with complementary capabilities to match the distinct computational characteristics of these workloads' heterogeneous kernels. However, existing multi-engine DL accelerator design approaches lack a systematic methodology, leaving fundamental questions unresolved. These include how to co-design engines for workloads with diverse computational characteristics and which engine combinations minimize aggregate execution costs (such as time or energy) across such workloads. Addressing these questions requires efficient exploration of exponentially large design spaces. To address these questions systematically, this work proposes Multi-Engine DL Accelerator Design Methodology (MEDEM). MEDEM defines generic engine abstractions, co-designs candidate instances (engines), and selects a combination of co-designed engines to minimize aggregate execution cost given diverse DL workloads and a resource budget. MEDEM encompasses a set of design strategies that efficiently navigate the exponentially large design spaces of engine co-design and combination selection, identifying highly optimized multi-engine accelerators. A comprehensive evaluation demonstrates that MEDEM identifies accelerators that outperform state-of-the-art designs, delivering geometric-mean improvements of up to 4.84x in energy-delay product (EDP) and 1.59x in throughput. The improvements are achieved using different resource budgets, demonstrating MEDEM's scalability, and using 51 single- and multi-model DL workloads, demonstrating its generalizability.