Combining language models and genetic search improves program solving
Combining LLMs and Genetic Search for ARC-AGI-2
Neural and Evolutionary ComputingArtificial Intelligence
Summary
Some computer programs can solve tricky problems, but it’s hard to find the right one quickly. The authors show that using a language model to create starting programs, and then improving them with a genetic search method, can find better solutions for specific tasks. Without the language model’s help, the genetic search alone couldn’t find any solutions. This approach means combining smart guesses with trial and error can work better than either alone.
What this means in practice
- •For software developers: Generate initial code sketches for hard programming tasks and improve them automatically within limited computing resources.
- •For automated testing teams: Enhance automated program repair tools by seeding genetic search with language model–generated candidate patches.
Authors
Val Dyachenko
Abstract
LLMs can generate programs for ARC-AGI-2 tasks, but the provided compute only allows a small number of attempts to generate, debug and validate solutions. Genetic algorithms can search and test many more programs, but random search rarely starts in a useful neighborhood of the solution space. We combine the two methods through a compact domain specific language (DSL). First, a quantized Qwen3.5-4B LLM generates an initial set of programs for each ARCAGI-2 task. Then, we use those programs to seed an initial population of starting programs, and use genetic algorithms to evolve these programs towards a solution to the given task. The DSL is designed such that every mutated program remains valid and can be executed. The initial programs proposed by the LLM solve 2 (3.3%) of the first 60 tasks of the ARC-2 public evaluation set. The genetic algorithm solves an additional 4, giving 6 correct test outputs in total (10.0%). If we try using evolving solutions without this LLM seeding, we do not arrive at any solutions at all. The results show that genetic search can improve programs generated by LLMs and produce additional correct solutions.