ForkSCOPE maps data analysis choices with human and AI help
ForkSCOPE: Charting the Agentic Garden of Forking Paths
Human-Computer Interaction
Summary
Data analysis can lead to many valid ways of reaching conclusions because researchers make many choices during the process. These choices can affect the results, but understanding all the possible paths is hard. The authors created ForkSCOPE, a system where humans and AI work together to organize and explore many different analysis routes without pre-set categories. This lets people see how different decisions impact outcomes and verify the analysis steps more easily.
What this means in practice
- •For data science teams: Manage and explore numerous alternative analysis pipelines on shared datasets to better understand result variation.
- •For software engineers: Integrate interactive tools that organize diverse code-based workflows to improve transparency and debugging.
Authors
Arjun Balaji, Batuhan Duru Yeltekin, Tian Zheng
Abstract
Even with a fixed dataset and research question, data analysis involves many defensible decisions. Understanding how these choices influence the results is scientifically important but remains challenging. Crowdsourcing and agentic AI can generate hundreds of end-to-end analyses, but scaling generation alone can create a processing bottleneck and an analytic ``black hole.'' A common workaround is to impose a shared fixed decision taxonomy, which can limit insight and understate uncertainty. We present ForkSCOPE, a human-AI collaboration framework that induces structure bottom-up from the code corpus of end-to-end analyses, without a taxonomy fixed before or after generation, so the organization and evaluation of the garden can scale with the corpus. ForkSCOPE surfaces the charted garden of forking paths through a human-AI collaboration pipeline and an evidence-linked interactive viewer for steering and verification: it spotlights organically identified forks and structures and produces a derived taxonomy and decision map compatible with existing multiverse tools.