CoRA-NAS improves neural network design with low-cost ranking and refinement
CoRA-NAS: Coarse Ranking and Anchor-Residual Refinement for Neural Architecture Search
Machine LearningComputer Vision and Pattern Recognition
Summary
Designing the best neural network architecture for a task is often expensive because it requires training many candidates. The authors created CoRA-NAS, a method that quickly ranks architectures by combining simple measures of their structure with a brief look at their early training performance. This approach selects promising architectures almost as well as if they had been fully trained, but with far less computation. Their method works reliably across different architecture search spaces and achieves high correlation with true performance without needing fully trained data for learning.
What this means in practice
- •For machine learning engineers: Quickly select high-quality neural network architectures across multiple search spaces with minimal training cost.
- •For automated machine learning platform teams: Integrate an efficient two-stage ranking and refinement stage to improve neural architecture selection without extensive computational resources.
Authors
Yifan Yang, Zhaoyan Wang, Zheng Gao, Xiaoyu Li, Jiaojiao Jiang
Abstract
Zero-cost proxies rank architectures cheaply, but their reliability varies across search spaces. We introduce CoRA-NAS (COarse Ranking + Anchor-residual), a two-stage framework combining a static ranking prior with low-cost learning-curve refinement. CoRA-Rank aggregates capacity and structure-at-initialization proxies through an equal-weight log-rank consensus and a target-free consensus gate. CoRA-Refine samples anchors across this prior, extrapolates their early validation curves, and propagates a learned residual correction with an ExtraTrees model. The refinement uses approximately 1% of the cost of fully training the candidate set. Fully trained architecture-accuracy labels are not used to fit the ranker. One configuration is used across spaces, with space-specific architecture encodings. Across NAS-Bench-201, NAS-Bench-101, TransNAS-Bench-101, and NATS-SSS, CoRA-Refine achieves mean Spearman correlations of 0.946, 0.715, 0.786, and 0.894, respectively. Its worst-space correlation of 0.715 is the highest among the compared methods. On NAS-Bench-201/CIFAR-100, its selected architecture reaches 73.32% accuracy, near the reported ground-truth best of 73.37%. On the pure size space, refinement recovers the static prior's shortfall relative to parameter count, while remaining tied with the strongest capacity proxies within noise. The resulting framework combines cross-space ranking robustness with low-cost architecture selection.