Large language models improve data fixes by mixing repair parts
From Outcomes to Components: A Counterfactual Evaluation Framework for LLM-Proposed Repairs
Artificial IntelligenceComputation and Language
Summary
Fixing broken data often means deciding which parts to change and how much. The authors studied how combining these parts from AI-suggested fixes can work better than changing them separately. They tested this in traffic systems and databases where errors are hidden in sums or groups. Sometimes fixing only one part made things worse, but fixing both together made things right again. They also showed that mixing parts from different AI fixes can recover most of the best possible improvement.
What this means in practice
- •For traffic system engineers: Improve traffic data corrections by testing combined target and update fixes from language models for more accurate network adjustments.
- •For database administrators: Enhance automated database repairs by selecting and recombining parts of AI-suggested fixes to better correct data errors obscured in aggregates.
Authors
Hengyu Ye, Songhua Hu
Abstract
An LLM-generated repair action specifies which records to change and by how much. A failed action may still contain a useful target set or numerical update, depending on how the two are paired. We evaluate this problem in traffic networks and databases, where aggregate observations obscure the records that need correction. We examine the pairings by executing each combination from the same initial state, using a reference action that restores the affected records. In 76 of 114 traffic actions, the gain from correcting both parts exceeds the sum of the gains from correcting each part alone. Six of these actions become worse after either correction alone, but improve when both parts are corrected. In database repair, replacing only the proposed update with the reference update worsens 17 of 108 model actions, all of which select unaffected records. We also test whether parts of two proposals can be reused. On held-out observations, selecting among the proposals and their recombined target-update pairs recovers 61.6% of the reference action's loss reduction, compared with 58.0% from selecting between the proposals alone.