Less cuts neural architecture search time to minutes with strong accuracy
LESS: Lightweight Evolutionary Supernet Search in Minutes
Neural and Evolutionary ComputingMachine Learning
Summary
Finding the best design for a neural network can be slow and costly because it requires testing many options carefully. The authors created a method called LESS that quickly narrows down good designs by using a brief training step and smart search techniques. This method evaluates candidate networks efficiently, achieving high accuracy on standard image recognition tasks much faster than comparable methods. LESS also works well without extra tuning on different datasets and larger search spaces, showing its versatility.
What this means in practice
- •For deep learning engineers: Cut neural network design search times from hours to minutes while maintaining high accuracy, enabling faster model development cycles.
- •For automated machine learning tool developers: Integrate lightweight evolutionary supernet search to quickly identify competitive architectures without lengthy training sessions in NAS frameworks.
Authors
Aviral Gandhi, Jinglue Xu, Jialong Li, Hitoshi Iba
Abstract
Low-cost NAS must both explore high-performing architectures and identify them reliably, yet reducing evaluation cost often weakens the fidelity of candidate comparisons. Training-free methods reduce evaluation cost by replacing learned task feedback with proxy signals measured at initialization. We introduce LESS (Lightweight Evolutionary Supernet Search), a data-driven method that combines a brief fair hard-path warm-up with discrete search under a single CMA-ES distribution. Each proposal is evaluated as its decoded hard genotype after six candidate-conditioned supernet updates. On NAS-Bench-201, LESS achieves \(93.189\pm0.467\%\) CIFAR-10 test accuracy in 409.1 seconds, coming within 0.04 percentage points of FairNAS using approximately \(1/24\) of its source-reported search time. Matched controls show that calibration improves selected validation accuracy by \(0.577\) percentage points while changing best-visited accuracy by only \(0.054\) points, indicating that its primary effect is to reduce selection regret. The frozen configuration transfers without tuning to CIFAR-100 and ImageNet16-120 with \(69.615\pm1.139\%\) and \(43.720\pm1.697\%\) accuracy. Applied without tuning to the larger DARTS space, LESS achieves \(96.95\pm0.14\%\) on CIFAR-10 and \(82.43\pm0.80\%\) on CIFAR-100, with each search completing in approximately 43.5 minutes on a single GPU. Together, these results show that short, balanced, data-dependent updates enable competitive neural architecture search across datasets and search spaces within minutes.