AI automation could rapidly speed up AI research and development
What if automating AI R&D triggers an intelligence explosion?
Computers and SocietyArtificial Intelligence
Summary
AI systems are starting to write much of the code they use to improve themselves. This could lead to a very fast growth in AI abilities, where progress that usually takes years happens in just a few months or less. The authors analyze evidence pointing to this possibility and warn that such rapid progress could bring big benefits but also serious risks, including losing control over AI systems. They recommend that policymakers watch these developments closely and plan how to manage this fast change.
What this means in practice
- •For government policy teams: Develop monitoring tools and frameworks to oversee AI automating its own research processes.
- •For corporate ai strategy teams: Plan for rapid AI capability growth by integrating risk management and governance tailored to self-improving AI.
A position paper. It proposes an approach and reports no results.
Authors
Alan Chan, Christoph Winter, Andrew Barto, Jakub Pachocki, Geoffrey Hinton, Eric Horvitz, Yoshua Bengio, Dawn Song, Jack Clark, Hilary Greaves, Anton Korinek, Samuel Hammond, Thore Graepel, Ben Bariach, Philip H. S. Torr, Sheila A. McIlraith, Jeff Clune, Sam Manning, Girish Sastry, Tom Davidson, Daniel Eth, Sören Mindermann
Abstract
In contrast to even a year ago, AI systems now write most of the code inside the companies that build them. As more of the AI research and development (R&D) pipeline is automated, could AI progress radically accelerate in an "intelligence explosion," where years of advances are compressed into months or less? Preliminary evidence suggests that it could. In this work, we assess this evidence, analyze an intelligence explosion's potential impacts, and propose policy responses. AI systems are on track to automate most AI R\&D work within a few years, and possibly all of it. If this triggers an intelligence explosion, it could dramatically bring forward AI's benefits, but also pose extreme risks: capabilities growth could accelerate far beyond what society can keep up with, humanity could lose control over superhuman AI systems, and checks on power within and between states, companies, and branches of government could be severely eroded. Although there remains much uncertainty about these possibilities, the high stakes warrant serious further attention. Policymakers should urgently obtain more visibility into the automation of AI R&D, develop ways to steer and constrain an intelligence explosion, and prepare society to adapt to an intelligence explosion's impacts.