
This repo contains a low-rank adapter for LLaMA-7b
fit on the [Stanford Alpaca](https://github.com/tatsu-lab/stanford_alpaca) dataset.

This version of the weights was trained with the following hyperparameters:

- Epochs: 10 (load from best epoch)
- Batch size: 128
- Cutoff length: 512
- Learning rate: 3e-4
- Lora _r_: 16
- Lora target modules: q_proj, k_proj, v_proj, o_proj

That is:

```
python finetune.py \
    --base_model='decapoda-research/llama-7b-hf' \
    --num_epochs=10 \
    --cutoff_len=512 \
    --group_by_length \
    --output_dir='./lora-alpaca-512-qkvo' \
    --lora_target_modules='[q_proj,k_proj,v_proj,o_proj]' \
    --lora_r=16 \
    --micro_batch_size=8
```

Instructions for running it can be found at https://github.com/tloen/alpaca-lora.
{"license": "mit", "datasets": ["yahma/alpaca-cleaned"], "duplicated_from": "tloen/alpaca-lora-7b"}
dataset:yahma/alpaca-cleaned license:mit region:us
TAGS
#dataset-yahma/alpaca-cleaned #license-mit #region-us 


This repo contains a low-rank adapter for LLaMA-7b
fit on the Stanford Alpaca dataset.

This version of the weights was trained with the following hyperparameters:

- Epochs: 10 (load from best epoch)
- Batch size: 128
- Cutoff length: 512
- Learning rate: 3e-4
- Lora _r_: 16
- Lora target modules: q_proj, k_proj, v_proj, o_proj

That is:



Instructions for running it can be found at URL
TAGS
#dataset-yahma/alpaca-cleaned #license-mit #region-us 

TAGS
#dataset-yahma/alpaca-cleaned #license-mit #region-us 
