Multi agent system improves accuracy of natural language SQL queries
NL2SQL with Iterative Self-Refinement
DatabasesArtificial Intelligence
Summary
Turning spoken or written questions into database commands is tricky because computers often make mistakes interpreting the user's intent or the database structure. The authors created a system that uses several cooperating AI ‘agents’, each with a specific role, to catch and fix errors. These agents check if the question matches the topic, understand the database layout, consider relevant performance indicators, fix syntax errors by running the query, and verify meaning by converting the query back into words. Their method made database queries from language much more reliable, especially in precise areas like insurance.
What this means in practice
- •For enterprise data teams: Generate more accurate SQL queries from natural language questions in complex domains with strict business rules.
- •For insurance analytics teams: Improve reliability in extracting insights from legacy insurance databases by aligning queries with domain KPIs and regulations.
Authors
Anjali Vemula, Akshat Dasula
Abstract
Bridging the gap between human intent and machine logic remains a critical challenge in data access for non-technical users, yet Large Language Models (LLMs) remain unreliable: they hallucinate schema elements, generate syntactically invalid queries, or fail to capture user intent. These limitations are especially critical in domains such as insurance, where KPIs, regulatory constraints, and legacy relational systems demand high precision. We introduce a multi-agent, persona-driven NL2SQL framework that combines (1) domain relevance detection, (2) schema-grounded knowledge construction, (3) KPI-aware SQL synthesis, (4) syntax-level self-repair using sqlite execution feedback, and (5) semantic consistency checking via SQL-to-NL reverse engineering. Our system significantly reduces query hallucination and improves semantic fidelity compared to direct LLM prompting. Evaluated on 60 queries (50 from a HuggingFace text-to-SQL dataset and 10 custom domain queries), the system achieves 90% correctness using Mistral across seven coordinated LLM calls showing that this approach can substantially increase the reliability of NL2SQL pipelines in high-stakes environments.