Peer review needs new rules as ai writes and checks papers
Ignore the Writing: A Proposal for Peer Review in the Age of AI
Human-Computer InteractionDigital Libraries
Summary
It used to be that a carefully written research paper showed real work behind it. Now, because AI can easily produce polished papers, this signal is broken. The authors propose a new way forward where short summaries are judged by people, detailed evidence is checked by AI, and humans decide the research direction. This approach tries to keep research trustworthy even when AI is involved in writing and reviewing.
What this means in practice
- •For conference organizers: Set submission formats to brief summaries plus machine-verifiable appendices to better cope with AI-generated papers and improve review quality.
- •For journal editors: Use AI to verify experimental data in supplement materials while relying on human reviewers to assess the research’s significance and guide the field.
A position paper. It proposes an approach and reports no results.
Authors
Vassilis Kostakos
Abstract
The written paper has long stood in as proof of work: a fluent, well-structured manuscript was taken as evidence that good research lay behind it. Generative AI has broken that link. Polished manuscripts now cost nothing to produce, submissions are outgrowing what reviewers can absorb, and venues are turning to AI for help. The logical endpoint is a loop where AI writes the papers and AI reviews them. We argue that this loop is survivable, but only if it is given a reward function, and that human reviewing effort should become that reward function. We propose that submissions should consist of a 1500-word brief plus an unlimited, machine-verifiable appendix. Human reviewers judge whether a pursuit is worthwhile, models verify the experimental record in the appendix, and human curation decides where the field goes next.