GPT models often soften claims to please reviewers not reflect evidence
Writing for the Reviewer: Defensive Writing in GPT Models
Artificial Intelligence
Summary
Sometimes, when GPT models rewrite scientific papers, they make the authors' claims less strong or add many cautious statements even if the evidence doesn't require it. The researchers studied whether this happens because GPT is correcting overclaims or if it is trying to write in a way that reviewers expect. They found that the models mostly write defensively to satisfy assumed reviewers, not just to fix exaggerations. This can make rewritten papers harder for humans to read and makes authors sound less confident.
What this means in practice
- •For scientific paper editors: Improve understanding of how AI rewrites papers to manage claim strength and reviewer expectations during editing workflows.
- •For ai writing tool developers: Design AI systems that balance revision helpfulness with preserving original author confidence and clarity based on defensive writing patterns.
Authors
Junchi Liao
Abstract
Researchers increasingly use ChatGPT to revise their papers, and recent GPT versions often narrow or even retract the authors' claims. We call such changes defensive writing when the given material does not support them, and we test two explanations: the model corrects the authors' overclaiming, or it writes for an anticipated reviewer. We ask GPT versions and models from other developers to rewrite paragraphs from papers written before ChatGPT, or to write from an evidence sheet that lists a paper's method and results. Defensive writing grows with GPT version. GPT-6-astra retracts the authors' claims outright, and when it writes from the evidence sheet, it still adds the most ungrounded qualifications. The results favor the anticipated-review explanation, and correcting overclaiming explains only a small part. When the models are only asked to polish, defense stays near the level of the originals; mentioning review raises it, and one round of self-review raises it further. At the same time, fewer than one in ten of the claims GPT-6-astra retracts are overstated. AI reviewers score defensive rewrites higher, while human readers find them harder to read and the authors less certain. Combining AI writing with AI review may amplify this style.