Papers for

interface designers

Papers whose findings have a practical use for this group, as judged from the abstract. Open a paper to read what it means in practice.

Agent simulation predicts human workload before task engagement

Synthetic TLX: Forecasting Human Workload Using Agent Simulation

Abstract: Assessing human workload for technology-mediated tasks helps prevent task failure caused by poor technology design. Traditionally, workload is assessed retrospectively using the NASA Task Load Index (TLX) after humans complete a task. What if we could forecast workload before a human attempts a task using agent simulation? We introduce Synthetic TLX, a new paradigm for proactive workload estimation that predicts NASA TLX scores for a given task, unlocking novel interaction opportunities and evaluation methods. To understand its viability, we conducted three experiments comparing human and agent-generated scores to evaluate where they align and diverge. We found agent estimates align with human scores particularly when prompted with a human persona and active task simulation. However, agents and humans diverge in the sources of workload they are sensitive to. Based on our findings, we present three applications to showcase Synthetic TLX's potential and discuss the future of workload-aware human-AI interaction.

Thu 10 SeptHuman-Computer Interaction
The gist
Measuring how hard a task feels to a person usually happens after they finish it, which can be too late to improve things. This paper presents Synthetic TLX, a method that uses computer agents to predict how much effort a person will feel before starting the task. The researchers tested if these agent-generated predictions match what people actually report. They found agents give similar workload estimates when given a specific human-like role and detailed task simulation, although they focus on different causes of workload compared to humans.
Open → 2609.12273v1

Large study reveals how people use multimodal AI task helpers

Large-Scale User Behavior Analysis in Multimodal AI-Assisted Manual Task Execution

Abstract: Conversational Task Assistants (CTAs) are multimodal dialogue systems that support users in complex real-world tasks such as cooking and DIY through voice, text, image, and video interactions. Prior user studies have focused on controlled settings, leaving limited understanding of real-world CTA usage at scale. In this work, we present a large-scale study of CTA usage based on thousands of users in-the-wild. Our large-scale real-world data analysis unveils new understandings of (i) user-CTA interaction flows, (ii) user intents, (iii) user conversational traits, and (iv) behavioral factors associated with user satisfaction. Our findings reveal key opportunities for future research in CTAs, particularly in user interaction design and task engagement, concluding with concrete design guidelines.

Mon 7 SeptHuman-Computer InteractionArtificial IntelligenceComputation and Language
The gist
It can be hard to understand how people really use AI helpers for complex tasks like cooking or fixing things because studies often happen in limited, controlled places. The authors analyzed data from thousands of real users interacting naturally with a conversational AI that helps through voice, text, pictures, and video. They found patterns in how people talk to the AI, what they want to do, how they behave, and what leads to them feeling satisfied. These insights help improve the design of future AI helpers so they can be easier and more engaging to use.
Open → 2609.07594v1