Human LLM chat datasets differ widely shaping AI user models
Dataset Signatures in Human-LLM Interactions and User Modeling
Computation and Language
Summary
Different collections of human-chat with AI conversations show noticeable differences that are easy for AI to identify. The authors found these differences remain even after sorting data by common categories, meaning there are hidden factors that make each dataset unique. These unique traits affect how AI systems learn about users and evaluate user interactions. Choosing which dataset to train AI on can change how user models and AI helpers perform in real situations.
What this means in practice
- •For ai developers: Select conversation datasets based on identified dataset signatures to improve user model training reliability.
- •For product managers: Assess how dataset choice affects user AI assistant performance evaluations to make informed deployment decisions.
Authors
Joseph Suh, Serina Chang
Abstract
Human--LLM interaction datasets shape our understanding of AI use and provide a foundation for downstream research, including training and evaluation of user models. In recent years, a growing number of datasets have sought to capture a representative picture of human--LLM interactions. But how different are the pictures these datasets provide, and what do those differences mean for research built on them? We study these questions across seven conversation datasets, spanning in-the-wild chat logs and human preference data. We begin by revisiting the dataset classification experiment of Torralba & Efros and find that neural network classifiers identify the source of a conversation from user messages alone well above chance, indicating distinctive dataset signatures. This separability persists after matching datasets on the dimensions of human-designed taxonomies, implying subtle differences that these taxonomies do not capture. We then examine the implications for user modeling: how dataset signatures propagate to the outputs of user models trained on these datasets; how dataset choice influences evaluations of user model quality and subsequent evaluations of LLM assistants paired with these user models; and how dataset classifiers can guide data selection for training user models. While each dataset is meant to capture a slice of 'real-world' interactions, our findings reveal the extent to which these slices diverge, and the consequences of those differences for research built on these foundations.