Stylometric features detect GPT-assisted student writing effectively
Detecting GPT-Assisted Writing Using Interpretable Stylometric Features
Computation and LanguageComputers and Society
Summary
It can be hard to tell when students get help from AI like ChatGPT in their writing. This study shows that by looking closely at word choices and grammar, computers can tell if someone wrote an essay alone or used AI help. The researchers tested different computer programs and found one that was good at spotting AI-assisted writing. This means it’s possible to detect AI help just by analyzing the writing itself.
What this means in practice
- •For academic integrity teams: Detect GPT-assisted student writing submissions by analyzing lexical and grammatical features with interpretable machine learning models.
- •For content moderation teams: Identify AI-assisted text in user-generated content to evaluate authenticity using stylometric feature-based classifiers.
Authors
Rajesh Kumar, Nabeel Siddiqui, Alexander Fuchsberger
Abstract
Distinguishing GPT-assisted from independently authored student writing has become a critical challenge in academia. This paper evaluates the discriminative capability of interpretable stylometric features extracted solely from submitted text. Using data from 90 participants who wrote both independently and with ChatGPT assistance, we evaluate eight machine learning classifiers while keeping data from the same participant together during validation. On the held-out test set, Random Forest achieved an ROC-AUC of 0.87 and an F1-score of 0.84, with False Positive and False Negative rates of 22.2% and 11.1%, respectively. SHAP analysis shows that lexical and grammatical characteristics drive the resulting predictions. The findings suggest that transparent, text-intrinsic features provide measurable signal for detecting GPT-assisted writing.