Dynamic clocking boosts energy efficiency of tensor processing units
Dynamic Slack-Aware Clocking for Near-Threshold Tensor Processing Units (TPUs)
Hardware Architecture
Summary
Running special AI chips called Tensor Processing Units (TPUs) at very low power levels saves energy but can cause errors because the timing of operations becomes unpredictable. The authors propose a new method that watches how long individual operations take and adjusts timing for each one instead of using a single slow clock for everything. This method uses simple predictors to group operations by speed and delays some cycles locally to keep things safe. Their tests show this approach improves energy efficiency without greatly hurting accuracy or requiring major hardware changes.
What this means in practice
- •For hardware engineers: Enable design of energy-efficient TPUs by dynamically adjusting operation timing without changing global clock frequency.
- •For data center operators: Reduce energy consumption of AI inference hardware by deploying TPUs with dynamic slack-aware clocking methods.
Authors
Muhammad Usman Nadeem, Sanghamitra Roy, Koushik Chakraborty
Abstract
Operating Tensor Processing Units (TPUs) in the near-threshold computing (NTC) region significantly reduces energy consumption but introduces high delay sensitivity to process variation and data activity. Conventional designs typically rely on a conservative, fixed global clock to ensure safety, which leaves large portions of timing margin unexploited as most operations finish well before the clock edge. We propose Dynamic Slack-Aware Clocking (DSAC), a proactive framework that replaces worst-case timing with operation-specific adjustments. DSAC employs lightweight Hamming-Distance, Most-Significant-Bit, and Hybrid predictors to estimate the delay sensitivity of individual multiply-accumulate (MAC) operations and classify them into three timing tiers. These tiers are enforced locally via dummy-hold cycles under a fixed global reference clock, enabling fine-grained timing adaptation without global clock retuning or frequency scaling. A closed-loop feedback controller monitors timing violations and updates tier thresholds at runtime to maintain resilience. Experiments on quantized DNN benchmarks demonstrate that the MSB predictor maintains high inference accuracy, with an average loss of only 1% even at aggressive performance points. Furthermore, DSAC achieves up to 1.55X better energy efficiency at 2.15X frequency scaling compared to a baseline TPU, while incurring an area overhead as low as 13%.