AI software factory speeds data system coding and operations by three times
Towards an AI Software Factory for Data Systems
Artificial IntelligenceDatabasesSoftware Engineering
Summary
Writing software involves many steps beyond just coding, and improving only one step only helps so much. The authors built an AI-powered software factory that speeds up all parts of the software life cycle, including planning, coding, reviewing, and operations. This system learns from ongoing work to continuously improve itself. They tested it in many Microsoft projects focused on data systems and found it made engineers work three times faster and handled code more efficiently. Some challenges remain for wider adoption.
What this means in practice
- •For software engineers: Improve end-to-end software development speed for data-focused projects by using an AI system that automates targeting, coding, review, and operations tasks.
- •For cloud operations teams: Use AI-generated metadata to enhance system management and automate operational improvements in data system deployments.
Authors
Anna Pavlenko, Bogdan Crivat, Brandon Haynes, Carlo Curino, Fotis Psallidas, Jaro Slawinski, Johannes Freischuetz, Laura Pereira Sanchez, Markus Weimer, Mathieu Demarne, Matthias Jasny, Mauktik Gandhi, Max Bovykin, Mirco Milletari, Purbasha Ghosh, Qiushi Bai, Raghu Ramakrishnan, Rahul Pandita, Sergiy Matusevich, Shivaram Venkataraman, Subru Krishnan, Md. Tareq Mahmood, Tiemo Bang, Venkatesh Emani, Xuan Zhao, Yiwen Zhu
Abstract
AI-assisted coding tools deliver significant acceleration of coding, but only limited impact across the end-to-end software development lifecycle (SDLC)--an Amdahl's law effect! In this paper, we discuss our progress towards building an AI SW Factory that accelerates all the stages of SDLC-Targeting, Coding, Reviewing, and Ops. The AI SW Factory produces a metadata exhaust that enables self-improvement by fine-tuning model weights and updating our World Model (a rich data substrate). We focus on Data Systems and the important class of Evolutionary Coding Tasks (i.e., those with a measurable objective to hill-climb) and report on 1) scaled deployments at Microsoft (tens of repositories) leading to 3x engineering efficiency above agentic coding and up to 22x token efficiency, and 2) several open challenges.