Papers for

database engineering teams

Papers whose findings have a practical use for this group, as judged from the abstract. Open a paper to read what it means in practice.

DataTrace benchmarks evidence-based root cause diagnosis in databases

DataTrace: Benchmarking Data Agents for Evidence-Grounded Root-Cause Localization

Abstract: Database agents diagnose pipeline failures, but a correct root cause is trustworthy only when it is supported by evidence the agent inspected. Existing benchmarks usually grade the final query or answer but fail to distinguish an evidence-based diagnosis from an unsupported correct answer. We introduce DataTrace, a benchmark for evidence-grounded root-cause localization over typed graphs of database artifacts. DataTrace uses controlled fault injection to derive root causes and evidence paths within 83 runnable environments without human annotations, and scores root-cause correctness and coverage of the supporting path separately. We also introduce CoSE, a generator-solver protocol in which an external verifier admits reproducible faults and checks both the submitted answer and the inspected evidence. Our evaluation shows that seven proprietary models achieve average root-cause accuracy 48%, with a mean per-model conditional unsupported rate of 18%. Fine-grained analysis shows that this gap varies sharply with fault mechanism, dependency distance, and evidence modality. After in-loop training on a matched held-out batch, the share of grounded correct diagnoses rises from 83.7% to 94.6%, while root-cause accuracy rises from 29.2% to 36.7%. These results show that while final-answer accuracy can hide unsupported diagnoses, evidence coverage provides an actionable signal to improve database agents. Code and data are available at https://hkustdial.github.io/datatrace/.

Fri 13 NovDatabases
The gist
When databases break, it's important not only to find the problem but to show the exact evidence behind the diagnosis. The authors introduce DataTrace, a tool that creates test scenarios with known failures and checks if database agents can find and support the real cause. They show current models often give right answers without proof. DataTrace reveals where these models fall short and helps improve them by focusing on evidence, making database troubleshooting more reliable.
Open → 2611.18926v1

Falcon generates complex synthetic data for natural language SQL queries

FALCON: A Model and Dataset Agnostic Framework for Synthetic Data Generation for NL2SQL Pairs

Abstract: Relational databases are among the most widely deployed forms of structured knowledge, and natural language access to them requires grounding language onto schema entities and relations while handling the ambiguity inherent in how people phrase requests. Existing synthetic NL-to-SQL data generation methods largely ignore this ambiguity and produce oversimplified queries that fail to prepare models for the complexity of real-world structured knowledge access. We present FALCON, a framework that generates realistic, ambiguity-aware NL-to-SQL data matching the complexity of challenging real-world benchmarks, at low cost using compact open models. Our approach combines reserved-word SQL seeding and persona-based prompting to generate structurally complex queries, while alignment-based filtering preserves difficulty by distinguishing genuinely incorrect examples from complex but valid queries. Human evaluation confirms consistent high quality across model sizes, and our generated data exceeds existing benchmarks in both SQL complexity and natural language richness. Difficulty-stratified analysis shows models trained on FALCON data increasingly outperform baseline-trained models as query complexity increases, validating our pipeline's success in generating challenging training data. When combined with a small proportion of existing benchmark data, mixed training recovers performance on simpler queries while preserving these advantages on complex ones. The model- and database-agnostic design enables organizations to generate high-complexity NL-to-SQL training data locally without external APIs.

Fri 2 OctComputation and LanguageDatabasesMachine Learning
The gist
Accessing data stored in databases using everyday language can be tricky because people phrase questions in very different ways. The authors created a method called FALCON that makes lots of realistic, challenging examples of natural language questions paired with the correct database queries. Their method includes making complex SQL queries and adding ambiguous language to better prepare AI models. They show that training with these examples helps AI perform better on tough questions, while still doing well on simpler ones. FALCON works with any database or AI model without needing external services.
Open → 2610.03625v1