Long-term Measurements: Towards a Longitudinal Understanding of Human-AI Interactions
2026-08-03 • Artificial Intelligence
Artificial Intelligence
AI summaryⓘ
The authors explain that language models are becoming a regular part of people's daily lives and may influence how people think and behave over time. They warn that short-term testing of these models misses possible long-term effects on users' minds and social behavior. To address this, the authors suggest combining natural language processing (NLP) techniques with social science methods that study changes over longer periods. This approach can help track and guide how these models impact people, aiming to prevent negative effects and promote positive ones. They also propose monitoring user behavior during interactions to catch problems early rather than after the fact.
Language modelsLongitudinal risksNatural language processing (NLP)Behavioral changeSocial science measurementsDiachronic analysisHuman-model interactionAlignment frameworksSafety risks
Authors
Nicole Mitchell, Dhruv Agarwal, Maty Bohacek, Remi Denton, Roma Patel
Abstract
Language models have taken on the role of a very new type of technology, by virtue of their "human-ness" and rapid integration into users' daily lives. This combination of features can introduce longitudinal risks---cognitive, developmental and socio-affective changes in humans---that might not surface in short-term interactions, but can have lasting long-term effects on users. This forms the basis of a critical new mission for NLP: to pivot from static, short-term evaluations of text generations to long-term measurements of behavioral changes, towards a diachronic understanding of human-model interactions. In this work, we draw from measurements used in social science fields that are crucial to understand emergent phenomena in longitudinal data. We discuss how computational methods in the field of NLP need to be combined with such measurements, not only to understand long-term safety risks of human-model interactions, but to help steer model development towards positive rather than negative outcomes for users. This ability to model human behavioral shifts as a function of model interactions can facilitate online rather than post-hoc detection of problematic behaviors, and should be leveraged in alignment frameworks to mitigate long-term risks in users.