# llama-instruct


This dataset was used to finetune [Llama-2-7B-32K-Instruct](https://huggingface.co/togethercomputer/Llama-2-7B-32K-Instruct).
We follow the distillation paradigm that is used by [Alpaca](https://crfm.stanford.edu/2023/03/13/alpaca.html), [Vicuna](https://lmsys.org/blog/2023-03-30-vicuna/), [WizardLM](https://arxiv.org/abs/2304.12244), [Orca](https://www.microsoft.com/en-us/research/publication/orca-progressive-learning-from-complex-explanation-traces-of-gpt-4/)
— producing instructions by querying a powerful LLM, which in our case, is the [Llama-2-70B-Chat](https://huggingface.co/meta-llama/Llama-2-7b-chat-hf) model released by [Meta](https://ai.meta.com/llama/). 

To build [Llama-2-7B-32K-Instruct](https://huggingface.co/togethercomputer/Llama-2-7B-32K-Instruct), we collect instructions from 19K human inputs extracted from [ShareGPT-90K](https://huggingface.co/datasets/philschmid/sharegpt-raw) (only using human inputs, not ChatGPT outputs).
The actual script handles multi-turn conversations and also supports restarting and caching via a SQLite3 database.
You can find the full script [here](https://github.com/togethercomputer/Llama-2-7B-32K-Instruct/blob/main/scripts/distill.py), with merely 122 lines!

The output of this step is a jsonl file, each line corresponding to one conversation:
```
{"text": "[INST] ... instruction ... [/INST] ... answer ... [INST] ... instruction ... [/INST] ..."}
{"text": "[INST] ... instruction ... [/INST] ... answer ... [INST] ... instruction ... [/INST] ..."}
{"text": "[INST] ... instruction ... [/INST] ... answer ... [INST] ... instruction ... [/INST] ..."}
```

For more details, please refer to the [Github repo](https://github.com/togethercomputer/Llama-2-7B-32K-Instruct).

## Languages

The language of the data is entirely English.
language:en license:llama2 arxiv:2304.12244 region:us
{"language": ["en"], "license": "llama2"}
TAGS
#language-English #license-llama2 #arxiv-2304.12244 #region-us 

# llama-instruct


This dataset was used to finetune Llama-2-7B-32K-Instruct.
We follow the distillation paradigm that is used by Alpaca, Vicuna, WizardLM, Orca
— producing instructions by querying a powerful LLM, which in our case, is the Llama-2-70B-Chat model released by Meta. 

To build Llama-2-7B-32K-Instruct, we collect instructions from 19K human inputs extracted from ShareGPT-90K (only using human inputs, not ChatGPT outputs).
The actual script handles multi-turn conversations and also supports restarting and caching via a SQLite3 database.
You can find the full script here, with merely 122 lines!

The output of this step is a jsonl file, each line corresponding to one conversation:


For more details, please refer to the Github repo.

## Languages

The language of the data is entirely English.
# llama-instruct


This dataset was used to finetune Llama-2-7B-32K-Instruct.
We follow the distillation paradigm that is used by Alpaca, Vicuna, WizardLM, Orca
— producing instructions by querying a powerful LLM, which in our case, is the Llama-2-70B-Chat model released by Meta. 

To build Llama-2-7B-32K-Instruct, we collect instructions from 19K human inputs extracted from ShareGPT-90K (only using human inputs, not ChatGPT outputs).
The actual script handles multi-turn conversations and also supports restarting and caching via a SQLite3 database.
You can find the full script here, with merely 122 lines!

The output of this step is a jsonl file, each line corresponding to one conversation:


For more details, please refer to the Github repo. ## Languages

The language of the data is entirely English.
TAGS
#language-English #license-llama2 #arxiv-2304.12244 #region-us 
 # llama-instruct


This dataset was used to finetune Llama-2-7B-32K-Instruct.
We follow the distillation paradigm that is used by Alpaca, Vicuna, WizardLM, Orca
— producing instructions by querying a powerful LLM, which in our case, is the Llama-2-70B-Chat model released by Meta. 

To build Llama-2-7B-32K-Instruct, we collect instructions from 19K human inputs extracted from ShareGPT-90K (only using human inputs, not ChatGPT outputs).
The actual script handles multi-turn conversations and also supports restarting and caching via a SQLite3 database.
You can find the full script here, with merely 122 lines!

The output of this step is a jsonl file, each line corresponding to one conversation:


For more details, please refer to the Github repo. ## Languages

The language of the data is entirely English.
passage: TAGS
#language-English #license-llama2 #arxiv-2304.12244 #region-us 
# llama-instruct


This dataset was used to finetune Llama-2-7B-32K-Instruct.
We follow the distillation paradigm that is used by Alpaca, Vicuna, WizardLM, Orca
— producing instructions by querying a powerful LLM, which in our case, is the Llama-2-70B-Chat model released by Meta. 

To build Llama-2-7B-32K-Instruct, we collect instructions from 19K human inputs extracted from ShareGPT-90K (only using human inputs, not ChatGPT outputs).
The actual script handles multi-turn conversations and also supports restarting and caching via a SQLite3 database.
You can find the full script here, with merely 122 lines!

The output of this step is a jsonl file, each line corresponding to one conversation:


For more details, please refer to the Github repo.## Languages

The language of the data is entirely English.