Compile by Training: Turning Natural-Language Specifications into Local Neural Functions
2026-09-03 • Computation and Language
Computation and LanguageArtificial IntelligenceMachine Learning
AI summaryⓘ
The authors created a method called 'compile by training' that turns a natural language description into a small, reusable neural program. Instead of asking a large remote model to do every task, their method trains a tiny helper model using examples made by bigger teacher models. This small model can then work on its own quickly without needing the big model each time. They tested it on a hard benchmark and got good accuracy, though it takes a bit longer to prepare the small model. They also showed practical uses like website helpers, avatar control, and language translation.
neural functionnatural language specificationteacher modelsadaptersemantic accuracycompile timeneural networksprogram synthesisinteractive servicelanguage-controlled avatar
Authors
Yuntian Deng, Pengyu Nie, Stuart Shieber
Abstract
Many recurring text functions are easy to describe but difficult to implement with rules, while calling a large remote model for every input introduces repeated cost, latency, and dependency on a provider. We present compile by training, which turns a natural-language specification into a reusable neural function. At compile time, teacher models generate task-specific examples that are used to train a small adapter for a compact interpreter. The resulting function runs without the teachers and can be stored, versioned, and composed like ordinary software. On FuzzyBench-Hard, a subset on which the Program-as-Weights fast compiler produced no exact matches, compile by training reaches 83.6% semantic accuracy. This higher accuracy comes with a higher compile-time cost: roughly a minute rather than seconds for the fast compiler. We deploy the compiler in a public interactive service and demonstrate compiled functions in a multi-site website helper, a language-controlled 3D avatar, and a bidirectional English-Claudish translator.