Paper review · October 20, 2026
NL2SQL with iterative self-correction
NL2SQL with Iterative Self-Refinement
The authors added syntax error correction and SQL-to-natural-language back-translation checks to a multi-agent NL2SQL pipeline for the insurance domain, reaching about 90% accuracy on 60 queries.
Problem
Insurers keep years of accumulated data in relational databases, but staff who do not know SQL find it hard to query directly. When an LLM converts natural language to SQL, it can make up tables or columns that do not exist (hallucination), produce wrong syntax, or generate queries that do not match the intent.
Method
The authors built a synthetic insurance database with 5 tables (customers, policies, claims, agents and applications) and put a dictionary of KPI formulas and a schema dictionary with semantic descriptions and statistics into the prompt. The pipeline has five stages: (1) CoT-based domain relevance check, (2) knowledge generation covering the tables, columns and KPI formulas to use plus a step-by-step plan, (3) SQL generation, (4) a syntax correction loop that takes sqlite execution errors and fixes them, and (5) semantic verification that turns the SQL back into natural language and compares it with the original question using LLM-as-a-Judge. This process is repeated to refine the query. The model used was mistral-large-2411.
Results
- The authors report about 90% accuracy on 60 queries: 50 from a synthetic text-to-SQL dataset on HuggingFace and 10 KPI queries they wrote themselves.
- Simple lookup (Data Retrieval) tasks reached 100%, while analysis and reporting tasks that need multi-step aggregation and KPI calculation reached about 85%.
- Each query needs an average of 7 LLM calls and about 20 seconds of latency, making it slower than direct prompting.
Limitations
There are only 60 evaluation queries, and experiments used only one model (Mistral) on a synthetic database. The paper gives no quantitative comparison with direct prompting or existing methods, so the size of the improvement cannot be confirmed. It also does not explain in detail how accuracy was judged. Comparison with other models is left for future work.
Why it matters
This is useful for practitioners who want to add natural-language queries to an in-house database. It shows a practical design that combines injecting schema and KPI dictionaries, correction based on execution feedback, and back-translation checks, along with the resulting latency and call costs.
Key terms
- NL2SQL
- The task of turning a natural-language question into an SQL query that can be run on a database.
- SQL→NL back-translation check
- A method that turns generated SQL back into a natural-language description and checks whether its meaning matches the original question. It catches queries that run but do not match the intent.
- LLM-as-a-Judge
- An evaluation method that has an LLM judge whether two outputs match or how good they are.
Original
Authors: Anjali Vemula, Akshat Dasula · Posted 2026-10-16 · License CC BY 4.0 · Based on the full text
This review is an AI-written summary of the paper. Please treat the original paper as the reference.