SMART-MIG: A Learning Framework for Scalable and Energy-Efficient GPU Scheduling
2026-06-29 • Distributed, Parallel, and Cluster Computing
Distributed, Parallel, and Cluster Computing
AI summaryⓘ
The authors explain that new GPU technology lets smaller machine learning tasks run on parts of a GPU, which can save energy but might slow things down. To better manage this, they created SMART-MIG, a system that smartly divides GPUs and schedules jobs using advanced learning methods and smart rules. Their approach keeps the system manageable even when handling many jobs and GPUs, and it performs better in balancing energy use and job speed than older methods. They also provide math-based benchmarks to measure how efficient their system really is.
Multi-Instance GPU (MIG)Energy EfficiencyJob SchedulingMean-Field Multi-Agent Reinforcement LearningGPU PartitioningTardinessParallel ComputingHeuristic Algorithms
Authors
Wenqing Yu, Neel Karia, Tanvi Hisaria, Clifford Stein, Olivier Tardieu, Asser Tantawi
Abstract
The emergence of Multi-Instance GPU (MIG) technology enables us to run smaller machine learning models on partitions of a GPU rather than the entire device, thus improving utilization and reducing energy consumption, albeit with potential performance trade-offs. Meanwhile, the growing energy demands of GPU-equipped data centers motivate the development of online partitioning and scheduling schemes that not only ensure fast job processing but also achieve high energy efficiency. However, achieving energy-tardiness efficiency with manageable algorithmic complexity in large-scale scheduling remains a great challenge, due to the dual objectives of deciding on the GPU partitions and scheduling jobs onto the slices of the heterogeneous partitions. To address this challenge, we propose SMART-MIG, a parallel computing system that combines Mean-Field Multi-Agent Reinforcement Learning (MF-MARL) for large-scale MIG repartitioning with tailored heuristic algorithms for job scheduling. We demonstrate that the complexity of the repartitioning component remains constant even as the number of jobs and GPUs increases. We also establish theoretical lower bounds on energy consumption and tardiness to rigorously benchmark system performance. Finally, extensive experiments show that SMART-MIG improves the energy-tardiness efficiency by $18\%$ compared to its corresponding static-partitioning counterpart, while being only $27\%$ above the theoretical lower bound on energy consumption.