Papers for

product teams for business intelligence software

Papers whose findings have a practical use for this group, as judged from the abstract. Open a paper to read what it means in practice.

Multimodal agents struggle to fix and check flawed data visualizations

VisAudit: Evaluating Multimodal Agents for Visual Diagnosis and Repair

Abstract: Multimodal agents are increasingly used for data visualization tasks but remain limited in autonomous review. Unlike humans, they may fail to recognize when a visualization is incorrect, determine what to change, repair it without disrupting correct content, and verify whether the intervention succeeded. Existing benchmarks largely evaluate predefined individual capabilities such as chart generation, instruction-guided editing, or defect detection, and therefore do not capture this gap in autonomous review. We introduce VisAudit, a benchmark for evaluating visualization diagnosis, repair, and verification. Given a rendered chart and configurable auxiliary evidence, including its source data table, intended text summary, and visualization code, an agent iteratively diagnoses potential defects, modifies and executes visualization code, inspects execution and visual feedback, and determines when no further intervention is needed. VisAudit defines three tracks spanning diagnosed repair, autonomous repair, and open-world verification, and contains 1,900 flawed instances across 21 chart types and 10 flaw categories, together with 300 initially correct charts. We construct the benchmark through controlled perturbations of validated source visualizations, with systematic verification and human-aligned quality control to ensure that injected defects are well-defined and recoverable from the available evidence. Experiments with leading multimodal models reveal a substantial gap from reliable autonomous review: the strongest evaluated model fully recovers only $47.4\%$ of flawed charts in the autonomous-repair setting.

Thu 1 OctMachine Learning
The gist
Charts and graphs often contain mistakes, but current AI tools find it hard to detect and fix these errors on their own. The authors created VisAudit, a test system that gives AI agents flawed charts along with related data, then checks if they can spot and repair the problems correctly without breaking other parts of the visualization. They found that even the best AI model fixed less than half of the flawed charts entirely on its own. This shows there is still a big gap before AI can reliably review and repair visualizations like humans can.
Open → 2610.02399v1