Papers for

professional writing tool designers

Papers whose findings have a practical use for this group, as judged from the abstract. Open a paper to read what it means in practice.

Written justification reduces uncritical acceptance of AI suggestions in writing

Think Before You Accept: Can Written Justification Reduce Uncritical Uptake of AI Writing Suggestions?

Abstract: Generative AI can offer students useful feedback, but its value depends on judging which suggestions are accurate and relevant. Prior research shows that strategic friction during human-AI interactions can promote critical uptake, but how to effectively implement such friction in academic contexts remains unclear. We examine whether requiring students to justify decisions to accept or reject AI suggestions can mitigate uncritical uptake in academic writing. In a randomized experiment embedded in a course activity (N=129), students wrote a data analysis proposal, received mixed-quality AI revision suggestions, and decided whether to accept or reject them. Students required to provide written justifications were 24 percentage points less likely to adopt flawed suggestions (65% vs. 41%), with no reduction in acceptance of sound suggestions (81% vs. 86%). However, thematic analysis revealed superficial engagement in the justification task and gaps in metacognitive monitoring and domain knowledge.

Sun 20 SeptHuman-Computer Interaction

The gist

Using AI to suggest edits can help students, but sometimes they accept changes without thinking if they are good. The authors tested if making students explain why they accept or reject AI suggestions helps them catch bad advice. Students who wrote justifications accepted fewer flawed suggestions but still accepted good ones at similar rates. However, many justifications were shallow and students still lacked some self-monitoring skills and knowledge.

Open → 2609.23936v1