SQLoom improves text to SQL accuracy with set contrastive learning
SQLoom: Set-Contrastive Learning of Syntactic, Semantic, and Logical Dependencies for Text-to-SQL
DatabasesArtificial Intelligence
Summary
Turning natural language questions into database queries is tricky because SQL has strict rules that normal language models don't handle well. The authors propose SQLoom, a way to train language models to better understand SQL’s grammar, meaning, and logic by comparing sets of good and bad query parts. They also fixed some errors in an existing dataset and tested their method on several benchmarks, showing big improvements in accuracy and speed. This makes it easier and faster for computers to write correct SQL from questions.
What this means in practice
- •For database application developers: Build tools that convert user questions into accurate SQL queries faster and with fewer errors.
- •For business intelligence teams: Create streamlined natural language interfaces for querying company databases, reducing the need for SQL expertise.
Authors
Hao Fu, Degang Sun, Wan Wei, Jinxia Wei, Chun Long, Peng Gao
Abstract
Text-to-SQL translates natural language questions into SQL queries, enabling database access without writing SQL. Fine-tuning open-weight large language models (LLMs) is cost-effective, but existing methods treat SQL as ordinary text and optimize token-level likelihood without explicitly modeling its syntactic, semantic, and logical requirements. Thus, a single incorrect alias or comparison operator can invalidate a query while contributing little to training loss. We propose SQLoom, a constraint-aware fine-tuning framework with supervision at three levels: syntactic validity under SQL grammar and database schema, semantic alignment between query decisions and the question, and logical consistency across dependent clause decisions. SQLoom extends supervised fine-tuning with a unified set-contrastive objective favoring sets of correct outputs over minimally different incorrect ones at each level. LLM agents propose this supervision, which a SQL parser and execution on counterexample databases verify. We also release Spider-Refined, a training dataset correcting 378 flawed Spider annotations and enriching database schemas. Experiments on Spider, BIRD, and three perturbed Spider variants show that SQLoom improves execution accuracy for Qwen3-1.7B from 62.9% zero-shot to 88.7% on Spider development, reaching 89.2% on Spider test and 73.4% on BIRD development. It outperforms fine-tuning, reinforcement learning, and agent-based baselines with up to 70 billion parameters. On question perturbations, SQLoom loses at most 0.6 accuracy points versus up to 12.1 for agent-based methods. It generates SQL in a single pass at 0.42s per query, 23$\times$ faster than a GPT-4-based agent method. These results demonstrate the effectiveness of set-contrastive supervision over token-level training.