Platform enables live experiments on human and AI team collaboration
Pairit: A Platform for Live Experiments on Human-AI Collaboration
Human-Computer InteractionArtificial Intelligence
Summary
Organizing teamwork with AI is tricky because it's hard to test how people and AI can best work together and share tasks. The authors created Pairit, an online tool that lets researchers set up real-time experiments where humans and AI chat, share work, and make decisions in a controlled way. It uses a simple configuration file to design and run these experiments, tracking detailed interactions. The platform has been tested in actual studies and helps share and repeat these experiments easily.
What this means in practice
- •For human-computer interaction teams: Conduct live studies measuring how humans and AI communicate and coordinate in shared tasks using customizable experimental setups.
- •For customer support platform developers: Design and test AI-human chat workflows that share tasks and decisions in real time to improve support agent collaboration.
Authors
Harang Ju, Sinan Aral
Abstract
Organizational design in the era of artificial intelligence requires experimental methods that can test how human-AI groups coordinate, delegate, and make decisions. Programmable platforms coordinate live human-to-human sessions or real-time human-AI chat, but researchers cannot easily declare experiment protocols in which AI participants both communicate and act on shared work within one auditable configuration. Here we introduce Pairit, an online platform that facilitates the design, testing, and deployment of experiments that test human-AI organizational designs and interventions. Through a single YAML configuration file, researchers declare an executable experiment graph (pages, routing, randomization, matchmaking, chat, shared workspaces, server-hosted agents, surveys, timers, and custom HTML components) and combine any number of humans and AI agents in live sessions. We have validated the feasibility of the platform through multiple live deployments, including peer-reviewed published studies, capturing high-resolution process traces of communication, negotiation, and collaborative work in live human-AI dyads. By representing complex interactive protocols as standardized, auditable configuration files, Pairit provides reusable infrastructure for specifying, deploying, and sharing live human-AI organizational experiments.