Papers for

data analytics developers

Papers whose findings have a practical use for this group, as judged from the abstract. Open a paper to read what it means in practice.

Vision language models help automate interactive visual analysis tasks

WidgetVA: A Widget-Centric Framework and Benchmark for Agentic Visual Analytics

Abstract: Visual analytics (VA) enables sensemaking through interactive visualization, but effective analysis often requires experts to translate high-level intents into long sequences of interface operations and iteratively interpret visual feedback. We study whether modern vision-language models (VLMs) can take on this role as autonomous VA operators that observe the interface, plan multi-step exploration, execute interactions, and adapt based on intermediate visual feedback. To support systematic development and evaluation, we first introduce WidgetVA, a widget-centric agentic VA framework that standardizes interactive components as structured widgets with unified action (e.g., filter and zoom) and perception-query (e.g., selection summaries) APIs. This standardization supports two modes of system construction: wrapping an existing VA system to make it agent-operable without rebuilding it, and composing a new system from widgets as modular building blocks. To help agents coordinate across widgets rather than plan each interaction from scratch, each widget further packages reusable analytical workflows, giving agents more than a bare set of callable functions to plan over. Building on this framework, we present WidgetVABench, a benchmark of single- and multi-widget VA tasks that require agents to perform multi-step interactions to uncover evidence and produce verifiable results. Each task also provides fine-grained reference annotations so that WidgetVABench can score Answer, Reference Trace Similarity, and State separately rather than collapsing agent performance into one success score. Experiments across multiple VLMs show that our framework provides an effective scaffold for agentic VA, while the diagnostic measures expose persistent limitations for future work. The WidgetVA framework and WidgetVABench have been released in https://github.com/Hiverwin/widgetva.

Mon 21 SeptHuman-Computer InteractionArtificial Intelligence
The gist
Visual analytics lets people explore data through interactive graphics, but doing this usually takes experts a lot of clicking and interpreting. The authors created WidgetVA, a framework that turns parts of visual analysis tools into standard interactive pieces called widgets. This helps AI models plan and perform multi-step data explorations by working with these widgets like building blocks. They also made WidgetVABench, a set of tests for AI agents to prove how well they can use these widgets to analyze data. Their experiments show the framework helps, but there are still challenges for AI to fully automate such tasks.
Open 2609.24094v1