
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# llama70b-sft-lora-CCW_torkish_text

This model is a fine-tuned version of [meta-llama/Llama-2-7b-chat-hf](https://huggingface.co/meta-llama/Llama-2-7b-chat-hf) on the generator dataset.
It achieves the following results on the evaluation set:
- Loss: 1.2999

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 0.0001
- train_batch_size: 8
- eval_batch_size: 2
- seed: 100
- distributed_type: multi-GPU
- num_devices: 2
- gradient_accumulation_steps: 4
- total_train_batch_size: 64
- total_eval_batch_size: 4
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: cosine
- num_epochs: 3.0

### Training results

| Training Loss | Epoch | Step | Validation Loss |
|:-------------:|:-----:|:----:|:---------------:|
| 2.3146        | 0.05  | 5    | 2.1372          |
| 2.0273        | 0.09  | 10   | 1.9383          |
| 1.8786        | 0.14  | 15   | 1.8045          |
| 1.7489        | 0.19  | 20   | 1.7071          |
| 1.6838        | 0.23  | 25   | 1.6380          |
| 1.6141        | 0.28  | 30   | 1.5933          |
| 1.5718        | 0.33  | 35   | 1.5631          |
| 1.5445        | 0.37  | 40   | 1.5386          |
| 1.5322        | 0.42  | 45   | 1.5171          |
| 1.5089        | 0.47  | 50   | 1.4985          |
| 1.4784        | 0.51  | 55   | 1.4826          |
| 1.4821        | 0.56  | 60   | 1.4681          |
| 1.4656        | 0.61  | 65   | 1.4554          |
| 1.4431        | 0.65  | 70   | 1.4432          |
| 1.4259        | 0.7   | 75   | 1.4328          |
| 1.4142        | 0.75  | 80   | 1.4224          |
| 1.4153        | 0.79  | 85   | 1.4136          |
| 1.4068        | 0.84  | 90   | 1.4052          |
| 1.4005        | 0.89  | 95   | 1.3990          |
| 1.3919        | 0.93  | 100  | 1.3913          |
| 1.3843        | 0.98  | 105  | 1.3852          |
| 1.3705        | 1.03  | 110  | 1.3793          |
| 1.3452        | 1.07  | 115  | 1.3737          |
| 1.344         | 1.12  | 120  | 1.3694          |
| 1.3405        | 1.17  | 125  | 1.3639          |
| 1.3396        | 1.21  | 130  | 1.3598          |
| 1.3482        | 1.26  | 135  | 1.3556          |
| 1.3395        | 1.31  | 140  | 1.3517          |
| 1.3346        | 1.35  | 145  | 1.3475          |
| 1.3162        | 1.4   | 150  | 1.3440          |
| 1.3325        | 1.45  | 155  | 1.3403          |
| 1.3165        | 1.49  | 160  | 1.3373          |
| 1.3292        | 1.54  | 165  | 1.3343          |
| 1.3177        | 1.59  | 170  | 1.3310          |
| 1.3061        | 1.63  | 175  | 1.3283          |
| 1.3011        | 1.68  | 180  | 1.3257          |
| 1.3125        | 1.72  | 185  | 1.3233          |
| 1.2977        | 1.77  | 190  | 1.3209          |
| 1.3091        | 1.82  | 195  | 1.3187          |
| 1.2991        | 1.86  | 200  | 1.3163          |
| 1.3007        | 1.91  | 205  | 1.3145          |
| 1.3041        | 1.96  | 210  | 1.3128          |
| 1.2941        | 2.0   | 215  | 1.3112          |
| 1.2823        | 2.05  | 220  | 1.3103          |
| 1.2678        | 2.1   | 225  | 1.3086          |
| 1.2768        | 2.14  | 230  | 1.3076          |
| 1.2749        | 2.19  | 235  | 1.3065          |
| 1.2707        | 2.24  | 240  | 1.3054          |
| 1.2686        | 2.28  | 245  | 1.3046          |
| 1.2807        | 2.33  | 250  | 1.3040          |
| 1.2709        | 2.38  | 255  | 1.3034          |
| 1.2638        | 2.42  | 260  | 1.3028          |
| 1.2684        | 2.47  | 265  | 1.3022          |
| 1.2639        | 2.52  | 270  | 1.3017          |
| 1.2708        | 2.56  | 275  | 1.3012          |
| 1.2782        | 2.61  | 280  | 1.3009          |
| 1.2571        | 2.66  | 285  | 1.3006          |
| 1.2694        | 2.7   | 290  | 1.3004          |
| 1.2706        | 2.75  | 295  | 1.3003          |
| 1.2728        | 2.8   | 300  | 1.3001          |
| 1.2692        | 2.84  | 305  | 1.3001          |
| 1.2553        | 2.89  | 310  | 1.3000          |
| 1.264         | 2.94  | 315  | 1.3000          |
| 1.2728        | 2.98  | 320  | 1.2999          |


### Framework versions

- PEFT 0.7.2.dev0
- Transformers 4.37.0.dev0
- Pytorch 2.1.2+cu121
- Datasets 2.16.1
- Tokenizers 0.15.0
{"library_name": "peft", "tags": ["trl", "sft", "generated_from_trainer"], "datasets": ["generator"], "base_model": "meta-llama/Llama-2-7b-chat-hf", "model-index": [{"name": "llama70b-sft-lora-CCW_torkish_text", "results": []}]}
peft safetensors trl sft generated_from_trainer dataset:generator base_model:meta-llama/Llama-2-7b-chat-hf region:us