LabBook improves program discovery by managing experimental histories
LabBook: Harnessing Experimental History for Efficient LLM-Driven Discovery
Machine LearningArtificial Intelligence
Summary
Finding new computer programs with AI often means looking at a few examples, but this can miss helpful clues from all past attempts. The authors created LabBook, a smart note keeper that remembers everything tried before and picks out the most important parts when making new programs. This way, AI can use all past information without getting overwhelmed by too much detail. Their tests show LabBook finds better solutions more efficiently across many kinds of problems.
What this means in practice
- •For software engineering teams: Improve automated program generation tools by efficiently using all prior development attempts to produce better code iterations.
- •For operations research analysts: Enhance heuristic algorithm design for complex optimization tasks through better management of experimental insights.
Authors
Bo Yuan, Wenqian Ye, Zelin Zhao, Lama Moukheiber, Henry Kautz, Aidong Zhang, Yongxin Chen
Abstract
Evolutionary approaches to LLM-driven discovery often generate new programs from a small set of selected ancestors. This keeps contexts manageable but can omit useful evidence from other experiments, whereas including the full experimental history produces long, redundant contexts. We introduce a simple, single-agent discovery harness built around LabBook, an agent-maintained memory that serves two complementary roles: guiding retrieval of relevant evidence from a complete experimental log and informing the generation of new solutions. At each iteration, the same agent combines its memory with retrieved evidence and jointly produces the next program and an updated LabBook. This separates complete history retention from selective context construction, without requiring an explicit population or branching search structure. On 49 Frontier-CS problems, LabBook improves the observed quality-cost trade-off over the evaluated evolutionary baselines with two backbones, while remaining competitive across nine additional mathematical, systems, and heuristic-design tasks. Code will be released at https://github.com/BoYuanVisionary/LabBook.