DraftTrace tracks writing and AI use to help understand student work
DraftTrace: A Multi-View Analytics Environment for AI-Integrated Writing
Computation and LanguageArtificial IntelligenceEmerging Technologies
Summary
It can be hard for teachers to see how students write when they use AI tools. The authors created DraftTrace, a system that shows the final writing, the steps the student took, and how they interacted with an AI helper. They tested it in a class and found it can tell the difference between typing text, copying AI-generated content, and other writing methods. This helps teachers understand the writing process better and see how students use AI at different times.
What this means in practice
- •For learning management system teams: Integrate multi-view writing analytics to improve feedback on student AI usage and writing behavior during assignments.$Commercial implications: Enables LMS providers to offer advanced AI-aware writing monitoring and evaluation features to educational institutions.
- •For educational software developers: Build tools that combine final text and writing process data to detect different student writing behaviors involving AI assistance.
Authors
Divyansh Chandarana, Sandipan De, Vivek Gupta
Abstract
Generative AI has changed how students produce writing assignments. The final artifact is no longer sufficient to understand the process through which it was produced. We introduce DraftTrace, a writing environment that jointly captures three complementary views of writing: the final product, the writing process and interactions with an integrated AI-assistant. DraftTrace reconstructs how a document develops over time and organizes these signals into submission, longitudinal, and class-level analytics for instructors. We deployed DraftTrace in a graduate NLP course with 81 students and compared their sessions with LLM-generated responses entered by automated tools and with copy-typed responses. While product measures distinguish differences in text formulation, process measures distinguish differences in how text is entered. Considering both views together helps characterize cases such as copy-typing. Interaction traces show that students use the assistant differently across stages of writing: to clarify the question at an early stage and to verify answers at a later stage. A preliminary instructor survey highlights the importance of multi-view writing analytics and their interpretability.