Vision models predict research paper acceptance from layout and content
PaperScorer: How Predictable Is Paper Acceptance?
Machine LearningComputer Vision and Pattern Recognition
Summary
Deciding whether a research paper gets accepted is complicated and depends on many factors. The authors created PaperScorer, a set of models that predict acceptance based only on the paper itself, using either its text or images of the pages. Surprisingly, models looking at the paper’s page images—capturing figures and layout—outperformed text-only ones. The system predicts very well for clearly good or bad papers but struggles with those in between.
What this means in practice
- •For conference program chairs: Assist in triaging submissions by highlighting papers likely to be accepted or rejected based only on content and presentation.
- •For academic publishers: Improve initial screening workflows using image-based models that capture paper layout and figures for acceptance prediction.
Authors
Sachin Konan, Jonathan Liu, Zhuang Liu
Abstract
Automated research is largely evaluated by benchmark performance, but the value of research rarely reduces to a single score. Humans judge a work's holistic contribution through peer review, which is hard to model because it is noisy and depends on many factors. However, what if, given only the paper, we could predict its peer-review decision directly? We introduce PaperScorer, a family of models that predict a paper's accept or reject decision from its extracted text or page screenshots, training only on the decision after removing author identity and formatting shortcuts. Vision models are the most predictive (68% accuracy on ICLR papers and 77% on ML papers from arXiv), beating text-only and frontier reasoning models. Unlike plain text, page images retain figures, layout, and presentation. This signal is robust, surviving smaller models and even compressing the whole paper into a single image. But accurate predictions are not evenly spread. PaperScorer is strongest on the lowest and highest rated papers, while papers rated in the middle stay unpredictable for our models and frontier systems alike. Additionally, its confidence is calibrated, so its most confident predictions are also its most accurate. Overall, PaperScorer shows how much of the decision a paper carries on its own, marking a first step toward AI-driven research evaluation.