Abstract

Natural language interfaces to databases have traditionally suffered from three structural limitations: exclusive targeting of relational SQL, unconditional dependence on large language model (LLM) inference at query time, and absence of any runtime signal when generated queries are semantically incorrect. This paper presents text2ql, an open-source Python framework that addresses all three limitations through a language-agnostic Intermediate Representation (QueryIR) and a pluggable renderer architecture. A single seven-stage detection pipeline serves both SQL and GraphQL targets; a zero-LLM deterministic mode delivers 100% execution accuracy at a median latency of 3.2 ms with no API cost; and every generated query carries a runtime confidence score in [0.15, 0.97] computed from an additive signal model. Evaluated on 50-query random samples from the Spider and BIRD benchmarks (indicative results; full-set evaluation is planned), the LLM-backed mode achieves 62–70% exact match and 84–91% execution accuracy; the deterministic mode achieves 100% execution accuracy with zero parse errors across all 100 test cases. An ablation study isolates schema-aware prompting as the dominant accuracy lever, contributing +18.4 percentage points of exact-match gain over the schema-free baseline on both benchmarks. text2ql is publicly available at pypi.org/project/text2ql under the Apache 2.0 license.

References

• W. A. Woods, "Progress in natural language understanding: an application to lunar geology," in Proc. AFIPS, 1973.
• G. G. Hendrix et al., "Developing a natural language interface to complex data," ACM TODS, vol. 3, no. 2, 1978.
• A. Popescu et al., "Towards a theory of natural language interfaces to databases," in Proc. ACM IUI, 2003.
• T. Scholak, N. Schucher, and D. Bahdanau, "PICARD: Parsing Incrementally for Constrained Auto-Regressive Decoding from Language Models," in Proc. EMNLP, 2021.
• D. Gao et al., "Text-to-SQL Empowered by Large Language Models: A Benchmark Evaluation," arXiv:2308.15363, 2023.
• F. Li and H. V. Jagadish, "NaLIR: An Interactive Natural Language Interface for Querying Relational Databases," in Proc. ACM SIGMOD, 2014.
• V. Zhong et al., "Seq2SQL: Generating Structured Queries from Natural Language using Reinforcement Learning," arXiv:1709.00103, 2017.
• J. Guo et al., "Towards Complex Text-to-SQL in Cross-Domain Database with Intermediate Representation," in Proc. ACL, 2019.
• B. Wang et al., "RAT-SQL: Relation-Aware Schema Encoding and Linking for Text-to-SQL Parsers," in Proc. ACL, 2020.
• B. Chen et al., "ShadowGNN: Graph Projection Neural Network for Text-to-Query," in Proc. NAACL, 2021.
• R. Cao et al., "LGESQL: Line Graph Enhanced Text-to-SQL Model with Mixed Local and Non-Local Relations," in Proc. ACL, 2021.
• M. Pourreza and D. Rafiei, "DIN-SQL: Decomposed In-Context Learning of Text-to-SQL with Self-Correction," in Proc. NeurIPS, 2023.
• X. Dong et al., "C3: Zero-shot Text-to-SQL with ChatGPT," arXiv:2307.07306, 2023.
• L. Zhang et al., "ACT-SQL: In-Context Learning for Text-to-SQL with Automatically-Generated Chain-of-Thought," arXiv:2310.17342, 2023.
• H. Li et al., "CodeS: Towards Building Open-Source Language Models for Text-to-SQL," in Proc. SIGMOD, 2024.
• Y. Zheng et al., "A Natural Language Interface for GraphQL Queries," in Proc. ICDE Workshop, 2021.
• P. Rai et al., "LLM-Assisted GraphQL Query Generation from Natural Language," arXiv:2402.09812, 2024.
• T. Yu et al., "Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Task," in Proc. EMNLP, 2018.
• J. Li et al., "Can LLM Already Serve as A Database Interface? A Big Bench for Large-Scale Database Grounded Text-to-SQLs," in Proc. NeurIPS, 2023.