user experience teams

Abstract: In human-LLM content co-generation, AI-generated information can enter final artifacts without being adequately processed by users, creating risks when artifacts are shared or acted upon. We study whether recognition-level uptake of atomic information units can be assessed in open-ended co-generation and predicted from low-cost interaction traces. We collected data from 62 participants across three tasks. For each final draft, we extracted atomic information units and generated post-task recognition questions, yielding 1187 unit-level uptake labels. We present TraceMind, which tracks units across Chat and Draft histories, aligns interaction traces with changing on-screen layouts, and models spatial, temporal, and workflow-informed evidence. TraceMind outperformed all learned baselines across AUROC, AUPRC-non, balanced accuracy, and macro-F1. We found that uptake unfolds throughout interaction, with sustained active engagement providing informative evidence beyond isolated signals. Our work shifts human-LLM co-generation from content adoption toward what users actually take up, motivating uptake-aware systems grounded in low-cost interaction traces.

Abstract: In real-world design practice, evaluations rarely rely on a single source of judgment. Designers routinely combine expert opinions, empirical studies, and computational models, each with distinct strengths and limitations. While machine learning offers methods to integrate multiple feedback sources, these approaches remain largely inaccessible to designers without technical expertise. In this paper, we explore how to integrate multiple feedback sources, primarily through: (1) a practical approach for multi-source integration, and (2) its implementation in MUSE, a no-code tool that allows designers to combine and balance diverse sources. Our technical findings show that independent modeling of multiple evaluation sources enables exploration across heterogeneous feedback, accommodates different evaluation speeds, surfaces disagreements between sources, and supports an adaptable evaluation setup that designers can reconfigure during their process. In a visualization design study, participants navigated their own judgments alongside simulator feedback, reporting a perception of enhanced confidence and flexibility. Our results highlight the viability of multi-source integration to support computational design, offering a step toward bridging the gap between advanced optimization methods and design practice.