TraceMind predicts how users absorb AI content during co-writing
TraceMind: Predicting User Information Uptake from Low-Cost Interaction Traces during Human-LLM Content Co-Generation
Human-Computer Interaction
Summary
AI can help people write content together, but users might not fully understand or remember the AI-generated parts. The authors created TraceMind to track small pieces of information and predict if users recognize them, using simple data from how users interact with the writing process. They tested TraceMind with 62 people and found it worked better than other methods by looking at how users engage over time. This work helps build tools that know what users actually understand, not just what they accept from AI.
What this means in practice
- •For content collaboration platforms: Improve co-authoring tools by detecting which AI-generated info users actually understand during document creation.$Commercial implications: Enables enhanced interactive AI writing assistants that ensure users comprehend generated content, increasing trust and usability.
- •For user experience teams: Monitor user engagement with AI content to design better interfaces that encourage active understanding rather than passive acceptance.
Authors
Yu Mei, Fengyou Zu, Ruiwen Zhang, Jie Cai, Chang Liu, Zhoutong Ye, Chun Yu, Yuanchun Shi
Abstract
In human-LLM content co-generation, AI-generated information can enter final artifacts without being adequately processed by users, creating risks when artifacts are shared or acted upon. We study whether recognition-level uptake of atomic information units can be assessed in open-ended co-generation and predicted from low-cost interaction traces. We collected data from 62 participants across three tasks. For each final draft, we extracted atomic information units and generated post-task recognition questions, yielding 1187 unit-level uptake labels. We present TraceMind, which tracks units across Chat and Draft histories, aligns interaction traces with changing on-screen layouts, and models spatial, temporal, and workflow-informed evidence. TraceMind outperformed all learned baselines across AUROC, AUPRC-non, balanced accuracy, and macro-F1. We found that uptake unfolds throughout interaction, with sustained active engagement providing informative evidence beyond isolated signals. Our work shifts human-LLM co-generation from content adoption toward what users actually take up, motivating uptake-aware systems grounded in low-cost interaction traces.