Efficient training method for edge CNN accelerators meets strict hardware rules
ShiftQAT: Quantization-Aware Training for the Numerical Contract of a Fixed-Function Edge CNN Accelerator
Machine Learning
Summary
Running deep learning models on small devices needs special adjustments to work well with tiny, efficient chips. The authors created a training method called ShiftQAT that ensures models fit the strict rules of a fixed-function edge chip, like limited number sizes and exactly one shift operation per layer. Their method trains networks so they naturally follow these constraints, avoiding accuracy drops seen in other approaches. When tested on real hardware, these models classify images quickly and with low energy use.
What this means in practice
- •For embedded system engineers: Design CNN models that strictly meet numerical limits of fixed-function edge accelerators to improve efficiency on low-power devices.
- •For mobile ai developers: Optimize image classification models for faster and more energy-efficient inference on edge hardware using the ShiftQAT training method.
Authors
Oguzhan Buyuksolak, Osman Erman Okman, Mehmet Gorkem Ulkar
Abstract
Fixed-function CNN accelerators make edge inference fast and energy-efficient, but they execute only networks that obey a precise numerical contract. On the MAX7800x family, this contract requires power-of-two, per-tensor, symmetric scales, 8-bit activations and biases, integer-only arithmetic, and one statically configured shift per layer. We present ShiftQAT, the quantization-aware training (QAT) flow we developed for this family, which produces networks satisfying the contract by construction. It is the default training path of ai8x-training, the family's open-source training toolchain. ShiftQAT resolves residual, concatenation, and weight-sharing dependencies so that activation scales propagate statically and each layer's requantization reduces to one offline-configured shift. It also trains weights and biases on a joint power-of-two grid that fits the 8-bit bias budget. Among the compared methods, only ShiftQAT satisfies all of the contract's scaling constraints, including signed 8-bit activation storage. Even so, on ImageNet it matches the W8A8 accuracy drop of less constrained QAT methods and stays within 2.9 percentage points of FP32 at W4A8. On MAX78002 silicon, ShiftQAT-trained MobileNetV2 and ResNet-18 classify ImageNet images in 14.6 and 8.4 ms at 5.1 and 2.8 mJ per inference.