
# Instruct_Phi2_Dolly15K
Fine-tuned from phi2，used Dolly15k for the dataset. 90% for training, 10% validation.  Trained for 2.0 epochs using QLora.  Trained with 1024 context window.

# Model Details
* **Trained by**: trained by HenryJJ.
* **Model type:**  **Instruct_Phi2_Dolly15K** is an auto-regressive language model based on the phi 2 transformer architecture.
* **Language(s)**: English
* **License for HenryJJ/Instruct_Phi2_Dolly15K**: apache-2.0 license


# Prompting

## Prompt Template With Context
chatml format

```
<|im_start|>system
{instruction}<|im_end|>
<|im_start|>user
{prompt}<|im_end|>
<|im_start|>assistant
```

## Prompt Template Without Context
```
<|im_start|>system
{instruction}<|im_end|>
<|im_start|>assistant
```

# Training script:
Fully opensourced at: https://github.com/hengjiUSTC/learn-llm/blob/main/trl_finetune.py. Run on 1 A10G instance for 4 hours.

```
python3 trl_finetune.py --config configs/phi2-dolly.yml
```
{"license": "apache-2.0", "datasets": ["databricks/databricks-dolly-15k"]}
transformers safetensors phi-msft text-generation conversational custom_code dataset:databricks/databricks-dolly-15k license:apache-2.0 autotrain_compatible endpoints_compatible region:us