Metacognitive scaffolds improve AI-assisted writing quality and recall
Metacognitive Scaffolds Create Productive Friction for AI-Assisted Knowledge Work
Human-Computer InteractionArtificial Intelligence
Summary
When people use AI to help with writing, they need to figure out how to ask the AI for help effectively. The authors show that giving writers special guidance to plan and break down their writing tasks helps them write better and remember more later. Interestingly, even though the writers found the help harder to work with, their results were better. This suggests that some challenge in using AI support can actually lead to better learning and work.
What this means in practice
- •For technical writers: Improve AI-assisted writing tools by adding guided prompts that help users plan and structure tasks, boosting output quality and memory retention.
- •For content marketing teams: Design AI interfaces that encourage users to interact with AI in ways that create productive challenges, improving content accuracy and depth over time.
Authors
Viktor Kewenig, Lev Tankelevitch, Sean Rintel, Advait Sarkar, Abigail Sellen
Abstract
Generative AI imposes new metacognitive demands on knowledge workers as their work shifts from production to planning, evaluation, and orchestration. In a randomized, between-subjects experiment on AI-assisted writing (297 workers, 594 essays), we test dynamically-generated metacognitive scaffolds supporting goal specification and task decomposition. We find that scaffolds improve writing quality, with this boost persisting in short-lag transfer when scaffolds are removed. We also find evidence of improved topic recall three days later. These effects are consistent with observed changes in how people learn to prompt AI systems after exposure to scaffolding. However, scaffolded participants rate the AI as less helpful and more cognitively demanding even as they perform better, suggesting that subjective ease can understate the value of support. Our findings make the case for metacognitive scaffolds in AI-assisted knowledge work, revealing a productive friction worth designing for as workers increasingly delegate work to agentic AI.