Evolving Executable Pipeline Programs for AutoML with Language Models
2026-08-17 • Machine Learning
Machine LearningNeural and Evolutionary Computing
AI summaryⓘ
The authors created LACE, a system that uses an evolutionary approach combined with a large language model to search for complete Python code pipelines for machine learning. Unlike traditional AutoML systems that only pick and tune preset components, LACE writes full programs that can be inspected and edited by users. They tested LACE on many standardized datasets and found it performs better than several popular AutoML tools and similarly to the best ones. The key advantage of LACE is producing flexible, reusable code pipelines rather than fixed models.
AutoMLpipelineevolutionary algorithmlarge language modelscikit-learnPython programmingtabular dataOpenMLmodel tuningmachine learning
Authors
Sofoklis Kitharidis, Cor J. Veenman, Jan N. van Rijn, Thomas Bäck, Niki van Stein
Abstract
Automated machine learning (AutoML) systems search for pipelines within a space of preprocessing operators, learners, and hyper-parameters specified in advance: they can select and tune known components, but cannot produce structure outside that space. We present LACE, an AutoML framework that instead searches over complete executable pipeline programs: an evolutionary loop maintains a population of scikit-learn-compatible Python classes, and a large language model acts as the variation operator. To our knowledge, LACE is the first to formulate general tabular pipeline AutoML this way, evaluated on standardized OpenML tasks under a leakage-controlled protocol that withholds dataset identity from the generator. Because every candidate is ordinary Python, the returned pipeline and the search that produced it can be inspected and edited directly, rather than only through a framework's model objects. On 68 OpenML classification tasks, LACE with GPT-5.4-mini significantly outperforms auto-sklearn, H2O, and a fixed XGBoost baseline, with no detectable difference against AutoGluon, the strongest search-based system evaluated, while covering the full benchmark. Newer tabular foundation models are more accurate on the subset of tasks they support, but apply a fixed pretrained predictor rather than returning an editable task-specific program. LACE's contribution is therefore not raw accuracy but a search space defined by code: complete coverage, pipelines practitioners can reuse directly, and a component set extended by editing the prompt rather than the framework.