An agent learns optimizer programs and shares them as text
Building the Harness Automatically: Self-Play in Code Distills a Text Harness for Black-Box Optimization
Machine Learning
Summary
Finding good solutions without knowing details is hard. The authors show a method where an agent practices creating and testing search strategies by writing small programs. It then summarizes what it learned as a short text called a harness, which helps improve performance of optimization tasks across different settings. This harness works with multiple language models and even on unseen challenges, offering a way to discover and share problem-solving strategies in a clear text form.
What this means in practice
- •For machine learning engineers: Use text-based harnesses to improve black-box optimization performance in tuning model hyperparameters on unfamiliar tasks.
- •For software developers: Integrate distilled search strategies as compact text programs to boost optimization utilities in general software tools.
Authors
Yi Wu, Zheng Ren, Zhiyu Hu, Haochen Wang, Daryl Chang, Li Wei, Ting Wang, Zhen Li, Pooja Gupta, Nitin Jindal, Lukasz Heldt
Abstract
Can an agent learn a numerical search strategy through executable practice and then transfer that strategy as text? We study low-budget black-box optimization, where unaided language models remain well below strong classical optimizers. During development, an agent repeatedly writes and evaluates optimizer programs. It then distills the resulting program and practice record once into a 197-word primary Harness A, which is frozen before evaluation. Harness A reduces Gemini Flash regret by 48\% in an independent $N=30$ study ($p<.001$), enters the GP-BO performance range on the practice family, and lowers mean regret on all three held-out BBOB landscapes. The same text improves every tested Gemini executor and transfers to Claude Sonnet, reducing regret by 43\% and 49\% ($p\leq.005$). An independent end-to-end replication produces Harness B, a different program and text at the same performance tier. The same framework also attains the lowest regret on a sealed YouTube reward-tuning production benchmark. Executable practice is thus a viable way to discover a search policy, and language a portable medium for deploying it.