metadata

license: mit

Fine-Tuned Google T5 Model for Text to SQL Translation

This repository contains a fine-tuned version of the Google T5 model, specifically trained for the task of translating natural language queries into SQL statements.

Model Details

Architecture: Google T5 (Text-to-Text Transfer Transformer)
Task: Text to SQL Translation
Fine-Tuning Datasets:
- sql-create-context Dataset
- Synthetic-Text-To-SQL Dataset

Fine-Tuning Datasets

sql-create-context Dataset:
- This dataset was created by modifying data from the following sources:
  - Zhong, Victor, et al. "Seq2SQL: Generating Structured Queries from Natural Language using Reinforcement Learning." (2017).
  - Yu, Tao, et al. "Spider: A large-scale human-labeled dataset for complex and cross-domain semantic parsing and text-to-SQL task." (2018).
- Citation:
```
@misc{b-mc2_2023_sql-create-context,
  title   = {sql-create-context Dataset},
  author  = {b-mc2}, 
  year    = {2023},
  url     = {https://huggingface.co/datasets/b-mc2/sql-create-context},
  note    = {This dataset was created by modifying data from the following sources: \cite{zhongSeq2SQL2017, yu2018spider}.},
}
```

Synthetic-Text-To-SQL Dataset:

A synthetic dataset for training language models to generate SQL queries from natural language prompts.

Citation:

@software{gretel-synthetic-text-to-sql-2024,
  author = {Meyer, Yev and Emadi, Marjan and Nathawani, Dhruv and Ramaswamy, Lipika and Boyd, Kendrick and Van Segbroeck, Maarten and Grossman, Matthew and Mlocek, Piotr and Newberry, Drew},
  title = {{Synthetic-Text-To-SQL}: A synthetic dataset for training language models to generate SQL queries from natural language prompts},
  month = {April},
  year = {2024},
  url = {https://huggingface.co/datasets/gretelai/synthetic-text-to-sql}
}

Ongoing Work

I am currently working to implement PICARD (Parsing Incrementally for Constrained Auto-Regressive Decoding from Language Models) to improve the results of this model. More details can be found in the original PICARD paper:

Citation:

@misc{scholak2021picardparsingincrementallyconstrained,
      title={PICARD: Parsing Incrementally for Constrained Auto-Regressive Decoding from Language Models}, 
      author={Torsten Scholak and Nathan Schucher and Dzmitry Bahdanau},
      year={2021},
      eprint={2109.05093},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2109.05093}, 
}