Students change how they engage and judge AI teammates over weeks
Out of sync: Accommodation, evaluation, and emergence in five weeks with an agentic AI teammate
Human-Computer InteractionComputers and Society
Summary
This paper studies how college students work with an AI teammate over several weeks. The AI acted either as a challenger to push thinking or as a supporter encouraging quieter members. The students changed how they spoke to the AI right away based on its role and adjusted how helpful they thought it was as they gained experience. Over time, the groups working with the supportive AI showed more positive feelings about the collaboration, even though they rated the AI as less helpful. The authors show that short-term tests miss how opinions and interactions develop with AI partners over time.
What this means in practice
- •For educational technology developers: Design AI teammates that adapt their social roles to improve student engagement over multiple sessions based on interaction changes.$Commercial implications: Enables development of AI-based team members that maintain engagement and collaboration effectiveness in educational software, improving user retention and outcomes.
- •For remote team software builders: Create AI assistants for virtual teams that modify their intervention style over repeated meetings to better support communication dynamics.
Authors
Nia Nixon, Jaeyoon Choi, Pedro Martins de Bastos, Mohammad Amin Samadi
Abstract
As agentic AI in education shifts from reactive tools toward partners that take initiative and persist over time, a central question is how learners relate to, evaluate, and benefit from artificial agency. We conducted a longitudinal experiment in an undergraduate learning analytics course, in which 17 student teams (N = 48) collaborated with a persistent LLM-based teammate across weekly sessions. The agent decided when to intervene through a trigger-based architecture, and its social role was manipulated between teams: a Cognitive Agent that challenged quick consensus or a Supportive Agent that invited quieter members to contribute. Combining chat discourse, post-session surveys, and spoken reflections, we identified distinct patterns unfolding across three timescales. Students accommodated their language to the agent's role immediately, using more social language toward the Supportive Agent from the first session. Their helpfulness evaluations were recalibrated from session to session in relation to the agent's observed behavior, rather than remaining anchored to students' pre-existing attitudes toward AI. A slower crossover emerged in perceived collaboration process and positive-affect word use in reflections: Supportive-Agent teams began below Cognitive-Agent teams and ended above them, although students rated the Supportive Agent less helpful throughout. Because these responses follow different trajectories, single-session evaluations of agentic AI may miss changes that emerge only through repeated interaction.