
This repository comes with LoRA checkpoint to make LLaMA into a chatbot like language model. The checkpoint is the output of instruction following fine-tuning process with the following settings on 8xA100(40G) DGX system.
- Training script: borrowed from the official [Alpaca-LoRA](https://github.com/tloen/alpaca-lora) implementation
- Training script:
```shell
python finetune.py \
    --base_model='decapoda-research/llama-7b-hf' \
    --data_path='alpaca_data_gpt4.json' \
    --num_epochs=10 \
    --cutoff_len=512 \
    --group_by_length \
    --output_dir='./gpt4-alpaca-lora-7b' \
    --lora_target_modules='[q_proj,k_proj,v_proj,o_proj]' \
    --lora_r=16 \
    --batch_size=... \
    --micro_batch_size=...
```

You can find how the training went from W&B report [here](https://wandb.ai/chansung18/gpt4_alpaca_lora/runs/nl1xi6ru?workspace=user-chansung18).
{"language": ["en"], "license": "apache-2.0", "tags": ["alpaca", "llama", "chat", "gpt4"], "pipeline_tag": "text2text-generation"}
alpaca llama chat gpt4 text2text-generation en license:apache-2.0 region:us
TAGS
#alpaca #llama #chat #gpt4 #text2text-generation #en #license-apache-2.0 #region-us 


This repository comes with LoRA checkpoint to make LLaMA into a chatbot like language model. The checkpoint is the output of instruction following fine-tuning process with the following settings on 8xA100(40G) DGX system.
- Training script: borrowed from the official Alpaca-LoRA implementation
- Training script:


You can find how the training went from W&B report here.
TAGS
#alpaca #llama #chat #gpt4 #text2text-generation #en #license-apache-2.0 #region-us 

TAGS
#alpaca #llama #chat #gpt4 #text2text-generation #en #license-apache-2.0 #region-us 
