
# WikiSQL Dataset (Reformatted for Generative Models)
This is the exact same dataset as WikiSQL: https://huggingface.co/datasets/wikisql, but with the data reformatted to allow direct use with text generation LLMs. The original license and credits for the original dataset remain in place.

Specifically, the changes from standard WikiSQL are:

1. The table details in WikiSQL were included as dictionaries but tools like [LangChain](https://python.langchain.com/en/latest/modules/chains/examples/sqlite.html) and [LlamaIndex](https://medium.com/llamaindex-blog/combining-text-to-sql-with-semantic-search-for-retrieval-augmented-generation-c60af30ec3b) build their prompts using a SQL DESCRIBE of the tables, which is included in this dataset as the table_info.

1. In addition, some of the SQL commands in WikiSQL that were not syntactically valid (e.g. due to identifiers not quoted) were removed. Specifically, we created in-memory (SQLite) tables using the SQL DESCRIBE of the tables, then ran the WikiSQL human readable SQL query against these in-memory tables. Any SQL queries that threw exceptions for any reason were discarded, and the rest that ran without exceptions were included in this dataset as the sql_cmd.

1. The SQL queries under sql_cmd were also formatted to capitalize keywords and do other pretty printing of the SQL using [SQLParse](https://sqlparse.readthedocs.io/en/latest/) to make the SQL more standard and easier to learn for smaller models.

# Suggested Uses

This dataset may be used for the following purposes:

1. Combine SQL queries with text based retrieval, using techniques like the [LlamaIndex SQLAutoVectorQueryEngine](https://gpt-index.readthedocs.io/en/latest/examples/query_engine/SQLAutoVectorQueryEngine.html).

1. Fine tuning LLMs to generate SQL commands from natural language inputs, given SQL DESCRIBE of tables and various rows. This is exactly the use case for the [LangChain](https://python.langchain.com/en/latest/modules/chains/examples/sqlite.html) SQLChain, so once fine tuned these LLMs may be used directly with these chains for theoretically better results (not tried at the time of writing)

1. Few shot prompt seeding of LLMs used to generate SQL commands from natural language inputs.

license:bsd-3-clause region:us
{"license": "bsd-3-clause", "dataset_info": {"features": [{"name": "input", "dtype": "string"}, {"name": "table_info", "dtype": "string"}, {"name": "sql_cmd", "dtype": "string"}], "splits": [{"name": "test", "num_bytes": 9526974, "num_examples": 15462}, {"name": "validation", "num_bytes": 5034756, "num_examples": 8243}, {"name": "train", "num_bytes": 33996901, "num_examples": 54963}], "download_size": 11329076, "dataset_size": 48558631}}
TAGS
#license-bsd-3-clause #region-us 


# WikiSQL Dataset (Reformatted for Generative Models)
This is the exact same dataset as WikiSQL: URL but with the data reformatted to allow direct use with text generation LLMs. The original license and credits for the original dataset remain in place.

Specifically, the changes from standard WikiSQL are:

1. The table details in WikiSQL were included as dictionaries but tools like LangChain and LlamaIndex build their prompts using a SQL DESCRIBE of the tables, which is included in this dataset as the table_info.

1. In addition, some of the SQL commands in WikiSQL that were not syntactically valid (e.g. due to identifiers not quoted) were removed. Specifically, we created in-memory (SQLite) tables using the SQL DESCRIBE of the tables, then ran the WikiSQL human readable SQL query against these in-memory tables. Any SQL queries that threw exceptions for any reason were discarded, and the rest that ran without exceptions were included in this dataset as the sql_cmd.

1. The SQL queries under sql_cmd were also formatted to capitalize keywords and do other pretty printing of the SQL using SQLParse to make the SQL more standard and easier to learn for smaller models.

# Suggested Uses

This dataset may be used for the following purposes:

1. Combine SQL queries with text based retrieval, using techniques like the LlamaIndex SQLAutoVectorQueryEngine.

1. Fine tuning LLMs to generate SQL commands from natural language inputs, given SQL DESCRIBE of tables and various rows. This is exactly the use case for the LangChain SQLChain, so once fine tuned these LLMs may be used directly with these chains for theoretically better results (not tried at the time of writing)

1. Few shot prompt seeding of LLMs used to generate SQL commands from natural language inputs.

# WikiSQL Dataset (Reformatted for Generative Models)
This is the exact same dataset as WikiSQL: URL but with the data reformatted to allow direct use with text generation LLMs. The original license and credits for the original dataset remain in place.

Specifically, the changes from standard WikiSQL are:

1. The table details in WikiSQL were included as dictionaries but tools like LangChain and LlamaIndex build their prompts using a SQL DESCRIBE of the tables, which is included in this dataset as the table_info.

1. In addition, some of the SQL commands in WikiSQL that were not syntactically valid (e.g. due to identifiers not quoted) were removed. Specifically, we created in-memory (SQLite) tables using the SQL DESCRIBE of the tables, then ran the WikiSQL human readable SQL query against these in-memory tables. Any SQL queries that threw exceptions for any reason were discarded, and the rest that ran without exceptions were included in this dataset as the sql_cmd.

1. The SQL queries under sql_cmd were also formatted to capitalize keywords and do other pretty printing of the SQL using SQLParse to make the SQL more standard and easier to learn for smaller models. # Suggested Uses

This dataset may be used for the following purposes:

1. Combine SQL queries with text based retrieval, using techniques like the LlamaIndex SQLAutoVectorQueryEngine.

1. Fine tuning LLMs to generate SQL commands from natural language inputs, given SQL DESCRIBE of tables and various rows. This is exactly the use case for the LangChain SQLChain, so once fine tuned these LLMs may be used directly with these chains for theoretically better results (not tried at the time of writing)

1. Few shot prompt seeding of LLMs used to generate SQL commands from natural language inputs.
TAGS
#license-bsd-3-clause #region-us 
 # WikiSQL Dataset (Reformatted for Generative Models)
This is the exact same dataset as WikiSQL: URL but with the data reformatted to allow direct use with text generation LLMs. The original license and credits for the original dataset remain in place.

Specifically, the changes from standard WikiSQL are:

1. The table details in WikiSQL were included as dictionaries but tools like LangChain and LlamaIndex build their prompts using a SQL DESCRIBE of the tables, which is included in this dataset as the table_info.

1. In addition, some of the SQL commands in WikiSQL that were not syntactically valid (e.g. due to identifiers not quoted) were removed. Specifically, we created in-memory (SQLite) tables using the SQL DESCRIBE of the tables, then ran the WikiSQL human readable SQL query against these in-memory tables. Any SQL queries that threw exceptions for any reason were discarded, and the rest that ran without exceptions were included in this dataset as the sql_cmd.

1. The SQL queries under sql_cmd were also formatted to capitalize keywords and do other pretty printing of the SQL using SQLParse to make the SQL more standard and easier to learn for smaller models. # Suggested Uses

This dataset may be used for the following purposes:

1. Combine SQL queries with text based retrieval, using techniques like the LlamaIndex SQLAutoVectorQueryEngine.

1. Fine tuning LLMs to generate SQL commands from natural language inputs, given SQL DESCRIBE of tables and various rows. This is exactly the use case for the LangChain SQLChain, so once fine tuned these LLMs may be used directly with these chains for theoretically better results (not tried at the time of writing)

1. Few shot prompt seeding of LLMs used to generate SQL commands from natural language inputs.
passage: TAGS
#license-bsd-3-clause #region-us 
# WikiSQL Dataset (Reformatted for Generative Models)
This is the exact same dataset as WikiSQL: URL but with the data reformatted to allow direct use with text generation LLMs. The original license and credits for the original dataset remain in place.

Specifically, the changes from standard WikiSQL are:

1. The table details in WikiSQL were included as dictionaries but tools like LangChain and LlamaIndex build their prompts using a SQL DESCRIBE of the tables, which is included in this dataset as the table_info.

1. In addition, some of the SQL commands in WikiSQL that were not syntactically valid (e.g. due to identifiers not quoted) were removed. Specifically, we created in-memory (SQLite) tables using the SQL DESCRIBE of the tables, then ran the WikiSQL human readable SQL query against these in-memory tables. Any SQL queries that threw exceptions for any reason were discarded, and the rest that ran without exceptions were included in this dataset as the sql_cmd.

1. The SQL queries under sql_cmd were also formatted to capitalize keywords and do other pretty printing of the SQL using SQLParse to make the SQL more standard and easier to learn for smaller models.# Suggested Uses

This dataset may be used for the following purposes:

1. Combine SQL queries with text based retrieval, using techniques like the LlamaIndex SQLAutoVectorQueryEngine.

1. Fine tuning LLMs to generate SQL commands from natural language inputs, given SQL DESCRIBE of tables and various rows. This is exactly the use case for the LangChain SQLChain, so once fine tuned these LLMs may be used directly with these chains for theoretically better results (not tried at the time of writing)

1. Few shot prompt seeding of LLMs used to generate SQL commands from natural language inputs.