Gricea platform helps build and share conversational AI studies
Gricea: An Open Science Platform for Conversational AI Research
Human-Computer InteractionArtificial Intelligence
Summary
Studying how people interact with conversational AI is important but often hard to repeat because details are scattered or missing. The authors created Gricea, a platform that packages these studies into clear, ready-to-run setups that anyone can use and modify. When tested, it recreated most published study setups and exposed common missing details that block exact copying. People from diverse backgrounds used Gricea to build new studies, showing it helps teamwork and repeated experiments. This approach could speed up learning about conversational AI by making research easier to share and build upon.
What this means in practice
- •For conversational ai developers: Create, share, and rerun conversational AI experiments using standardized setups to improve study consistency and accelerate improvements.
- •For user experience teams: Build and test conversational interfaces with repeatable experiments that can be inspected and extended by distributed teams.
Authors
Nikhil Sharma, Yunlin Gong, Xinyang Cheng, Ziang Xiao
Abstract
We need studies on conversational AI (CAI) at scale to understand human behavior and shape CAI design. However, fragmented reporting of systems and study configurations hinders replication, extension, and knowledge accumulation. We present Gricea, an open-science platform representing studies as configurable, deployable research artifacts that researchers can run, inspect, share, and reuse. Informed by a formative analysis of prior CAI research, Gricea couples study procedures, participant-facing systems, and conversational task behavior in. In a replication study using Gricea, we replicated configurations 93% of eligible CUI 2026 papers; while also flagging missing information in 96% of papers that hinder faithful replication --- further motivating Gricea's need. In a user study, researchers and practitioners from diverse backgrounds successfully constructed runnable studies addressing various open-ended research questions. Together, these findings demonstrate Gricea's support for constructing, reproducing, and extending CAI studies through shared research artifacts, enabling cumulative knowledge building through open science.