AI autonomously formulates and reviews research papers on laptop
Letting AI Formulate the Research Question: Autonomous Research Agents
Artificial IntelligenceComputers and Society
Summary
Many AI systems can help with research, but usually humans pick the topics. This paper studies an AI that chooses its own research questions and writes papers without human input. The AI also checks its work by having another AI review it, improving the quality. The authors tested this AI multiple times on a laptop, producing papers that external AI reviewers judged to be at workshop level. They provide their system and all data to enable others to reproduce the results.
What this means in practice
- •For software developers: Create AI assistants that independently generate and validate research projects without human subject selection.
- •For science communication teams: Use autonomous AI to draft workshop-level research summaries that require minimal human editing.
Authors
Ko Fujimura
Abstract
Systems such as The AI Scientist have shown that AI agents can carry out and write up research, but in those demonstrations humans chose the research area and selected the outputs to show. We study letting the AI formulate the research question itself, and we report every run. Our system gives an off-the-shelf coding agent an instruction document and two skills: one for formulating the question, in which the agent diverges from recent technologies into problems of other fields, screens its candidates for known failure modes and obtains a critique from a different model; and one for peer review, in which a reviewer from another model family reruns the study from its records before the agent revises the paper. Over 13 development runs, changes to the one or two sentences that set the research area coincided with large shifts in what the candidate questions and the chosen papers were about, which led us to the area of a new technology applied to a societal problem. We froze the system after a pilot run and report five consecutive runs: the pilot, counted by a disclosed amendment to the pre-registration, and four pre-registered runs. All five runs went from question formulation to a revised, re-computable paper, with local computation on a laptop and no research input from humans, and external AI reviewers reading only the papers judged them at or near workshop level. We release the system, the five papers and the records needed to reproduce them.