Detectors rely on formatting cues to spot machine generated scientific texts
Measuring Artifact Reliance in Machine-Generated Text Detectors
Computation and Language
Summary
It looks like computer programs can reliably tell if scientific papers are written by humans or machines. But this study found that many detectors are actually noticing small formatting differences, like how line breaks appear after extracting text from PDFs, rather than the writing itself. When the researchers fixed these formatting quirks, the detectors became less accurate. This means that some tools might be fooled if the formatting changes, not the content.
What this means in practice
- •For content moderation teams: Improve detection of AI-generated scientific texts by adjusting models to ignore formatting artifacts and focus on textual content.
- •For document processing engineers: Develop preprocessing steps that normalize extracted scientific texts to reduce misleading signals caused by extraction artifacts.
Authors
Anastasia Voznyuk, Andrey Grabovoy
Abstract
Detectors of machine-generated text score almost perfectly on scientific-literature benchmarks, implying we can reliably distinguish with these detectors human and machine-generated documents. We looked at ten scientific-document subsets from six popular resources, and on several of them that apparent ease is a side effect of how the documents were put together. A logistic regression using 27 superficial features reached 0.833-0.997 macro-F1, and on the three arXiv subsets it outperformed every detector we evaluate with F1-scores 0.951-0.997. The strongest cue to the detector is the PDF extraction artifact: human texts retain their original line breaks, whereas generated texts are extracted as a single block of prose. This cue, together with three other formatting features, tell the classes apart at 0.996 AUC. To distinguish benchmarks that contain such a signal from detectors that rely on one, we introduce format counterfactuals: just re-wrapping a machine-written abstract without changing a single word reverses 20-27% of a public detector's correct decisions. Normalizing formatting before training reduces the generalization gap from 0.32 to 0.22. On format-clean benchmarks, it also raises out-of-domain macro-F1 from 0.33 to 0.65 for detectors trained on artifact-containing subsets.