Adaptive calibration improves hardware aware neural architecture search efficiency
Coverage You Can Steer: Online Conformal Calibration for RL-Driven Hardware-Aware NAS
Machine Learning
Summary
Finding the best computer hardware design often takes a lot of time because each design must be tested thoroughly. The authors improved a method that predicts which designs are promising and worth testing, ensuring the predictions remain accurate even as the search evolves. They introduced an adaptive way to adjust these predictions on the fly, which lets the system prune less promising options without losing accuracy. This means hardware-aware neural architecture search can be done faster and more reliably than before.
What this means in practice
- •For hardware design engineers: Accelerate the evaluation of neural network architectures customized for specific hardware by dynamically pruning unlikely candidates without compromising performance.
- •For machine learning infrastructure teams: Optimize resource usage in automated architecture search by controlling prediction coverage online, reducing the computational cost of model training.
Authors
Pedro Brandimarte, Nerea Aranjuelo, Marcos Nieto, Oihana Otaegui
Abstract
Hardware-aware neural architecture search (NAS) is dominated by evaluation cost: every architecture must be trained before its reward is known. Conformal-prediction filters cut this cost by pruning candidates whose predicted-reward upper bound misses a threshold, with a distribution-free guarantee that at most a fraction $δ$ are wrongly discarded. That guarantee assumes exchangeability between calibration and test candidates, which the surrounding reinforcement-learning (RL) loop violates: the policy's proposals improve as search proceeds and, in layer-by-layer construction, shift within every episode. We replace one-shot quantile estimation with online feedback control (Adaptive Conformal Inference, with tuning-free, locally-adaptive, and group-conditional variants), restoring steerable coverage: dialing the target delivers it, monotonically and reproducibly, for arbitrary sequences. Across three neural-network architecture families and both single-step and sequential search (three seeds), it tracks every requested level to within ${\sim}10^{-3}$ while pruning 25-50% of evaluations at no measured accuracy cost, whereas static calibration loses control of its coverage and a Gaussian-process baseline stays conservative regardless of the request. Finally, used as an acquisition function on one constrained testbed, the same optimistic bound beats random search, a gain that fixed optimism already carries and online calibration sharpens. The source code is available at https://github.com/Vicomtech/rl-hw-nas.