GreCAPTCHA tests authors understanding to verify research ownership
greCAPTCHA: Assessing Understanding as Evidence of Research Authorship Under Generative AI
Digital LibrariesArtificial IntelligenceHuman-Computer Interaction
Summary
It can be hard to tell if a research paper was really written by the person named as the author, especially with AI tools helping to write text. The authors created greCAPTCHA, a test that asks questions about a research paper to see if the author really understands it well. When tried out with 31 researchers, the test could accurately tell which papers they wrote based on their answers. People who used it said the questions made sense and suggested improvements. This tool can help universities and journals check if authors truly know their own work.
What this means in practice
- •For academic conference organizers: Use greCAPTCHA to verify authors’ understanding and authorship of submitted research papers under supervised conditions.
- •For journal editorial teams: Integrate greCAPTCHA to assess whether submitted manuscripts were genuinely authored to uphold publication integrity.
Authors
Justin Payan, Bálint Gyevnár, Atoosa Kasirzadeh, Nihar B. Shah
Abstract
Conferences, journals, funders, schools, and universities are struggling with a surge of potentially AI-generated submissions from ostensibly human authors, who may not have exercised sufficient human oversight for their manuscripts. In turn, institutions evaluating submissions can no longer reliably credit expertise based solely on authors' names on submitted work. To address this problem, we propose greCAPTCHA, a proctored assessment approach that measures authors' understanding of research manuscripts via the construct of capacity to verify, which we define as the knowledge and reasoning required to critically assess the contents underlying one's contributions to a manuscript. greCAPTCHA generates questions assessing multiple levels of understanding and provides an evaluative report based on authors' responses. Using a prototype implementation, we conduct a user study and semi-structured interviews with $31$ researchers to evaluate greCAPTCHA. Its automated scores predict which papers were or were not authored by study participants with an AUC of $0.90$. Participants reported positive overall experiences with the system and remarked on the appropriate construct validity for author understanding, while also suggesting important changes to be made before deployment. Our results provide initial evidence that greCAPTCHA can assess manuscript-specific understanding under proctored conditions.