Mitigating Compiler Fusion-Induced Power Bursts in Mobile NPU Inference as the Battery Depletes
Hardware ArchitectureArtificial Intelligence
Summary
The authors studied how running AI tasks on a smartphone's neural processing unit (NPU) can cause sudden drops in voltage, leading the phone to slow down to protect itself. They found that certain ways the phone combines operations can create big bursts of power use that trigger this slowdown earlier than necessary. To fix this, they developed a technique that changes the AI task structure before running it to avoid these big power spikes. Testing on a Snapdragon 8 Gen 3 chip showed this method reduced peak power, kept the phone running smoothly at lower voltages, and only slightly increased the task time.
Authors
Ryoga Yuzawa, Masayoshi Tomizuka
Abstract
Mobile devices increasingly rely on real-time NPU inference for camera and perception workloads. Under low-voltage conditions, however, a single inference can induce an instantaneous voltage droop in the power-delivery network, causing the power management integrated circuit to invoke dynamic voltage and frequency scaling (DVFS) and increase latency. We present a measurement study of this effect on a commercial smartphone. We show that aggressive operator fusion in a mobile NPU compiler can create monolithic superlayers whose concentrated execution produces large peak-current bursts. These bursts shift the DVFS-onset voltage upward and reduce the low-voltage operating margin. We further evaluate a practical black-box mitigation: a measurement-guided, pre-compilation graph rewrite that inserts barriers at selected peak-to-average power ratio hot spots to prevent harmful superlayer merging in the vendor NPU compiler. On Snapdragon 8 Gen 3 with MobileNetV4 at 768 x 768 resolution on ImageNet-1k, this method reduces peak current from 3.12 A to 1.94 A with 3.76% latency overhead, preserves stable latency deeper into the low-voltage regime, and shifts the inferred DVFS margin by approximately 173 mV.