Autonomous software agents optimize scientific code far faster than humans
Is manual software optimization a thing of the past?
Software EngineeringArtificial Intelligence
Summary
As scientific programs handle bigger data, making them run faster usually needs experts who know a lot about programming and math. The authors studied whether AI agents can improve these programs on their own, even ones already optimized by humans. They tested the agents on three scientific tasks and found the AI-made versions ran up to 100 times faster while still being correct. This suggests that in some cases, people may no longer need to manually optimize software and can instead guide AI to do it automatically.
What this means in practice
- •For scientific software developers: Use autonomous AI agents to speed up mature scientific programs with minimal human tuning while ensuring correctness.
- •For bioinformatics engineers: Accelerate genomic data analysis tasks by applying AI-driven optimizations to existing gene set enrichment tools.
Authors
Pavlin G. Poličar, Martin Špendl, Tomaž Hočevar
Abstract
Scientific software is increasingly required to process larger datasets while maintaining acceptable execution times. Software optimization traditionally requires substantial expertise in programming, algorithms, and numerical methods. Recent advances in large language models (LLMs) offer the possibility of automating much of this process. We investigate whether LLM-based agents can autonomously achieve substantial performance improvements in scientific software, including mature implementations that have already been extensively optimized by human developers. We tasked an LLM-based agent with optimizing software for three computational problems: t-SNE, single-sample gene set enrichment analysis (ssGSEA), and graphlet counting. Humans defined the scope, correctness criteria, and a verification mechanism, after which the agent worked autonomously, in some cases for several hours. Code maintainers reviewed each resulting implementation and verified its correctness. The optimized implementations were faster in all tested configurations, by up to two orders of magnitude over the fastest existing tools. The improvements included low-level code optimizations, mathematical reformulations, and an entirely new algorithm for graphlet counting. Software optimization can increasingly be delegated to autonomous agents, with the human role shifting from implementing optimizations to deciding which software to optimize, defining objectives, providing verification mechanisms, and ensuring the correctness of the final software. For well-scoped, verifiable problems, we argue that manual software optimization may be a thing of the past.