AREAs-Lab: An Interactive Environment for AI-driven Requirement Elicitation for AI Systems
Human-Computer Interaction
Summary
The authors created AREAs-Lab, a tool that helps users write clear and complete requirements for AI systems by asking smart questions and looking at the data involved. They noticed that people often start with vague instructions, so their system works by gradually refining these instructions based on user answers. To test this, they built a set of example tasks from many different fields, including both the full correct instructions and the incomplete ones that the system tries to improve. They also made a way to automatically check how well the system’s questions lead to better requirements by simulating a user who only shares information when asked properly. This work helps study how AI assistants can better understand what users want to build AI systems that do the right things.
Building effective AI systems increasingly depends on writing high-quality task requirements, yet users often struggle to articulate the constraints, preferences, and edge cases that determine success. This problem is especially acute in AI development, where behavior is shaped not only by human expectations but also by data characteristics. We present AREAs-Lab, an interactive environment for AI-driven Requirement Elicitation for AI systems. In AREAs-Lab, an assistant iteratively refines an initially incomplete requirement by analyzing the underlying dataset and asking targeted clarification questions to uncover the user's latent intent. To study this setting systematically, we construct a synthetic benchmark grounded in 16 public datasets spanning diverse domains and task types. Each benchmark instance includes a user profile, a complete reference requirement, and an intentionally underspecified version that serves as the assistant's starting point. We further introduce an automated evaluation pipeline based on an AI-simulated user that reveals hidden information only when appropriately prompted, enabling scalable and reproducible assessment of interactive elicitation quality. AREAs-Lab provides a controlled testbed for studying how AI assistants can transform vague user goals into actionable requirements for AI systems.