Incremental instructions reduce language model story quality and structure
The Effects of Incremental Instruction Delivery on Language-Model Creative Writing
Computation and Language
Summary
The paper studies how giving large language models writing instructions all at once versus in small steps affects their creative stories. The authors found that when instructions are given bit by bit, the stories are less coherent and less well-structured, even if the models follow the rules equally well. They created a new way to measure how well stories keep both rules and narrative structure. Human judges agreed that stories done with all instructions upfront were generally better. This means writing tools should check not just if the instructions are followed but also if the story stays well put together.
What this means in practice
- •For interactive writing tool developers: Improve creative writing apps by designing interfaces that present all writing requirements upfront to preserve story coherence and structure.
- •For chatbot designers: Enhance multi-turn conversational agents by integrating measures of narrative quality when instructions are given incrementally.
Authors
Anshuman Singh, Abrar Eyasir, Haseeb Yaqoob, John Manavalan
Abstract
Large language models are increasingly used as interactive writing tools, where users develop stories, revise ideas, and introduce new requirements across multiple turns rather than specifying a complete brief upfront. Yet most evidence on multi-turn instruction degradation comes from tasks with objectively verifiable outcomes, leaving unclear whether incremental interaction harms creative artifacts in ways that explicit requirement checks cannot capture. We study this question using 160 human-authored creative-writing tasks across six genres, presenting each intended specification either upfront or progressively over 5-9 turns to six distinct open-weight model families, yielding 960 matched pairs. Progressive delivery reduces explicit constraint adherence and produces its largest writing-quality degradation in structure/coherence. The structural gap persists among outputs with equal observed adherence, suggesting that measured requirement loss alone does not explain the observed structural difference. We define Creative Integrity as a compact measure of joint adherence and narrative structure; under incremental delivery, models retain 71.2% of FULL Creative Integrity (95% CI [68.2%, 74.3%]). A three-rater human study over 50 matched pairs independently recovers FULL advantages in structure/coherence, craft, and genre effectiveness, while automated scores remain positively associated with aggregated human ratings. These findings show that interactive creative-writing systems should be evaluated not only on whether requirements survive conversation, but also on whether evolving requirements remain coherently integrated into the final artifact. Our dataset, benchmarks, and source code are available at: https://github.com/solusops/SISTER-2026-Team19