Genealogy guided evolution finds better neural network designs
EvoTreeNAD: Genealogy-Guided Evolution for LLM-Driven Neural Architecture Discovery
Neural and Evolutionary ComputingMachine Learning
Summary
Designing neural networks that work well usually needs experts to carefully choose and test many options, which takes a lot of time and computing power. The authors created EvoTreeNAD, a system that starts from scratch and builds a family tree of network designs by making small changes based on the best-performing ones. This method uses language AI to suggest new designs and write code for them, helping the system learn and improve over time without needing an initial blueprint. They showed that EvoTreeNAD finds network designs that outperform several existing methods on popular image tasks.
What this means in practice
- •For machine learning engineers: Automatically generate effective neural network architectures for image classification without predefined design spaces.
- •For healthcare data teams: Create customized neural networks that improve accuracy on medical image analysis tasks using automated architecture search.
Authors
Lishan Yu, Derek Jiu, Qizhen Lan, Xiaoqian Jiang
Abstract
AI-driven scientific discovery accelerates research by autonomously developing solutions and designs. Large language model (LLM) agents support this process through iterative generation and evaluation. Yet these iterations alone do not ensure cumulative progress or establish which directions to pursue next. Costly evaluation further constrains the scope of exploration. Neural architecture discovery brings these challenges together, coupling open-ended design with resource-intensive experimentation. We introduce EvoTreeNAD, a genealogy-guided evolutionary algorithm that constructs trainable architectures without a supplied seed or a hand-specified search space. Starting from an empty root, it grows a persistent genealogy in which each new node represents a complete architecture. Top-percentile values computed from each node and its descendants guide lineage selection. Using the selected design history, an Idea Agent proposes a variant and a Code Agent implements it. Each evaluated variant becomes a child node, expanding the genealogy while providing evidence for subsequent lineage selection. Our theoretical analysis establishes the existence of stationary variation regimes as the genealogy grows. Under specified variation assumptions, sustained top-percentile family values quantify the probability of generating high-reward architectures in these regimes. EvoTreeNAD discovers architectures that outperform the compared NAS and NAD baselines, achieving CIFAR-10/100 test errors of $2.05{\pm}0.06\%$ and $15.09{\pm}0.22\%$. On all six MedMNIST-v2 tasks, the discovered architectures surpass the strongest listed baselines. A controlled CIFAR-10 study further shows that EvoTreeNAD outperforms direct generation, best-of-$N$ greedy continuation, and full-family-mean routing.