generated_from_trainer dataset:anli base_model:meta-llama/Llama-2-13b-chat-hf region:us
---
base_model: meta-llama/Llama-2-13b-chat-hf
tags:
- generated_from_trainer
datasets:
- anli
model-index:
- name: sft-trl-claim-128-llama2-13b-chat-hf
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# sft-trl-claim-128-llama2-13b-chat-hf

This model is a fine-tuned version of [meta-llama/Llama-2-13b-chat-hf](https://huggingface.co/meta-llama/Llama-2-13b-chat-hf) on the anli dataset.
It achieves the following results on the evaluation set:
- Loss: 0.4327

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 5e-05
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 500
- num_epochs: 3.0

### Training results

| Training Loss | Epoch | Step   | Validation Loss |
|:-------------:|:-----:|:------:|:---------------:|
| 1.2823        | 0.02  | 1000   | 1.5153          |
| 1.1442        | 0.03  | 2000   | 1.4907          |
| 1.1578        | 0.05  | 3000   | 1.4674          |
| 1.1452        | 0.06  | 4000   | 1.4707          |
| 1.175         | 0.08  | 5000   | 1.4579          |
| 1.1516        | 0.1   | 6000   | 1.4382          |
| 1.2068        | 0.11  | 7000   | 1.4134          |
| 1.2737        | 0.13  | 8000   | 1.3936          |
| 1.3977        | 0.15  | 9000   | 1.4009          |
| 1.0338        | 0.16  | 10000  | 1.3899          |
| 1.0997        | 0.18  | 11000  | 1.3971          |
| 1.1361        | 0.19  | 12000  | 1.3488          |
| 1.012         | 0.21  | 13000  | 1.3628          |
| 1.0694        | 0.23  | 14000  | 1.3532          |
| 1.067         | 0.24  | 15000  | 1.3531          |
| 1.2775        | 0.26  | 16000  | 1.3492          |
| 1.1953        | 0.28  | 17000  | 1.3598          |
| 1.0807        | 0.29  | 18000  | 1.3294          |
| 1.2402        | 0.31  | 19000  | 1.2770          |
| 1.1454        | 0.32  | 20000  | 1.2803          |
| 1.0369        | 0.34  | 21000  | 1.2688          |
| 0.9862        | 0.36  | 22000  | 1.2651          |
| 1.151         | 0.37  | 23000  | 1.2371          |
| 1.1605        | 0.39  | 24000  | 1.2199          |
| 1.1367        | 0.41  | 25000  | 1.2268          |
| 1.011         | 0.42  | 26000  | 1.2123          |
| 1.1792        | 0.44  | 27000  | 1.1763          |
| 0.8839        | 0.45  | 28000  | 1.1784          |
| 1.166         | 0.47  | 29000  | 1.1999          |
| 1.2947        | 0.49  | 30000  | 1.2064          |
| 0.7324        | 0.5   | 31000  | 1.2174          |
| 0.9919        | 0.52  | 32000  | 1.1837          |
| 1.3371        | 0.54  | 33000  | 1.1761          |
| 0.9874        | 0.55  | 34000  | 1.1966          |
| 1.0411        | 0.57  | 35000  | 1.1885          |
| 1.1669        | 0.58  | 36000  | 1.1789          |
| 1.231         | 0.6   | 37000  | 1.1616          |
| 0.9799        | 0.62  | 38000  | 1.1003          |
| 1.0801        | 0.63  | 39000  | 1.0844          |
| 1.0984        | 0.65  | 40000  | 1.1037          |
| 1.1046        | 0.67  | 41000  | 1.0683          |
| 1.1537        | 0.68  | 42000  | 1.0569          |
| 1.0744        | 0.7   | 43000  | 1.0661          |
| 1.1451        | 0.71  | 44000  | 1.0460          |
| 0.9822        | 0.73  | 45000  | 1.0559          |
| 1.0439        | 0.75  | 46000  | 1.0473          |
| 0.8704        | 0.76  | 47000  | 1.0721          |
| 0.9934        | 0.78  | 48000  | 0.9789          |
| 1.134         | 0.79  | 49000  | 0.9800          |
| 1.0722        | 0.81  | 50000  | 1.0122          |
| 1.0894        | 0.83  | 51000  | 0.9872          |
| 0.822         | 0.84  | 52000  | 1.0067          |
| 1.0764        | 0.86  | 53000  | 0.9941          |
| 1.1809        | 0.88  | 54000  | 0.9966          |
| 1.0228        | 0.89  | 55000  | 0.9657          |
| 0.9944        | 0.91  | 56000  | 0.9778          |
| 0.7453        | 0.92  | 57000  | 0.9442          |
| 1.1951        | 0.94  | 58000  | 0.9621          |
| 0.7633        | 0.96  | 59000  | 0.9271          |
| 0.5546        | 0.97  | 60000  | 0.9493          |
| 1.2136        | 0.99  | 61000  | 0.9211          |
| 0.7377        | 1.01  | 62000  | 0.8821          |
| 0.8892        | 1.02  | 63000  | 0.8635          |
| 0.9491        | 1.04  | 64000  | 0.8695          |
| 0.9627        | 1.05  | 65000  | 0.8808          |
| 0.7405        | 1.07  | 66000  | 0.8450          |
| 1.1451        | 1.09  | 67000  | 0.8496          |
| 0.9935        | 1.1   | 68000  | 0.8708          |
| 1.2591        | 1.12  | 69000  | 0.8246          |
| 1.1467        | 1.14  | 70000  | 0.8376          |
| 0.8296        | 1.15  | 71000  | 0.8411          |
| 0.9733        | 1.17  | 72000  | 0.8356          |
| 1.1116        | 1.18  | 73000  | 0.8564          |
| 0.9909        | 1.2   | 74000  | 0.8420          |
| 1.0602        | 1.22  | 75000  | 0.8398          |
| 1.0284        | 1.23  | 76000  | 0.8429          |
| 0.9611        | 1.25  | 77000  | 0.8288          |
| 0.9866        | 1.27  | 78000  | 0.8432          |
| 0.6751        | 1.28  | 79000  | 0.8109          |
| 0.9637        | 1.3   | 80000  | 0.8039          |
| 1.2506        | 1.31  | 81000  | 0.8088          |
| 1.1821        | 1.33  | 82000  | 0.8080          |
| 0.9813        | 1.35  | 83000  | 0.8003          |
| 1.175         | 1.36  | 84000  | 0.7962          |
| 0.8377        | 1.38  | 85000  | 0.7913          |
| 1.114         | 1.4   | 86000  | 0.7977          |
| 0.9089        | 1.41  | 87000  | 0.7779          |
| 0.9896        | 1.43  | 88000  | 0.7665          |
| 0.7499        | 1.44  | 89000  | 0.7693          |
| 1.0132        | 1.46  | 90000  | 0.7420          |
| 0.6964        | 1.48  | 91000  | 0.7405          |
| 0.9243        | 1.49  | 92000  | 0.7396          |
| 0.8555        | 1.51  | 93000  | 0.7448          |
| 0.9978        | 1.52  | 94000  | 0.7449          |
| 1.2293        | 1.54  | 95000  | 0.7324          |
| 0.8886        | 1.56  | 96000  | 0.7519          |
| 0.8325        | 1.57  | 97000  | 0.7625          |
| 0.5212        | 1.59  | 98000  | 0.7554          |
| 0.8564        | 1.61  | 99000  | 0.7211          |
| 0.7141        | 1.62  | 100000 | 0.7227          |
| 0.5702        | 1.64  | 101000 | 0.6725          |
| 1.0332        | 1.65  | 102000 | 0.6611          |
| 1.0074        | 1.67  | 103000 | 0.6633          |
| 0.8829        | 1.69  | 104000 | 0.6682          |
| 0.5767        | 1.7   | 105000 | 0.6641          |
| 0.6375        | 1.72  | 106000 | 0.6652          |
| 0.7965        | 1.74  | 107000 | 0.6728          |
| 1.1104        | 1.75  | 108000 | 0.6564          |
| 0.5071        | 1.77  | 109000 | 0.6273          |
| 0.9045        | 1.78  | 110000 | 0.6298          |
| 0.7056        | 1.8   | 111000 | 0.6223          |
| 1.013         | 1.82  | 112000 | 0.6290          |
| 0.943         | 1.83  | 113000 | 0.6281          |
| 0.6309        | 1.85  | 114000 | 0.6231          |
| 0.7388        | 1.87  | 115000 | 0.6069          |
| 0.5067        | 1.88  | 116000 | 0.5786          |
| 1.144         | 1.9   | 117000 | 0.5723          |
| 1.1245        | 1.91  | 118000 | 0.5761          |
| 0.9093        | 1.93  | 119000 | 0.5832          |
| 1.1755        | 1.95  | 120000 | 0.5798          |
| 0.4415        | 1.96  | 121000 | 0.5789          |
| 0.8623        | 1.98  | 122000 | 0.5793          |
| 0.7605        | 2.0   | 123000 | 0.5529          |
| 0.9743        | 2.01  | 124000 | 0.5542          |
| 1.0476        | 2.03  | 125000 | 0.5585          |
| 1.0112        | 2.04  | 126000 | 0.5673          |
| 0.9344        | 2.06  | 127000 | 0.5644          |
| 0.976         | 2.08  | 128000 | 0.5771          |
| 1.0471        | 2.09  | 129000 | 0.5615          |
| 0.626         | 2.11  | 130000 | 0.5597          |
| 0.6866        | 2.13  | 131000 | 0.5520          |
| 0.7723        | 2.14  | 132000 | 0.5416          |
| 0.7605        | 2.16  | 133000 | 0.5407          |
| 0.8413        | 2.17  | 134000 | 0.5452          |
| 1.1015        | 2.19  | 135000 | 0.5506          |
| 0.7203        | 2.21  | 136000 | 0.5470          |
| 0.7008        | 2.22  | 137000 | 0.5535          |
| 1.035         | 2.24  | 138000 | 0.5404          |
| 0.8432        | 2.25  | 139000 | 0.5459          |
| 0.7886        | 2.27  | 140000 | 0.5403          |
| 1.1197        | 2.29  | 141000 | 0.5533          |
| 0.8474        | 2.3   | 142000 | 0.5237          |
| 0.8785        | 2.32  | 143000 | 0.5325          |
| 1.1119        | 2.34  | 144000 | 0.5105          |
| 1.2089        | 2.35  | 145000 | 0.5133          |
| 0.8626        | 2.37  | 146000 | 0.5097          |
| 1.106         | 2.38  | 147000 | 0.5120          |
| 0.9681        | 2.4   | 148000 | 0.5116          |
| 1.0139        | 2.42  | 149000 | 0.5102          |
| 0.6389        | 2.43  | 150000 | 0.5152          |
| 0.86          | 2.45  | 151000 | 0.5141          |
| 1.041         | 2.47  | 152000 | 0.5102          |
| 0.984         | 2.48  | 153000 | 0.4967          |
| 0.7444        | 2.5   | 154000 | 0.4792          |
| 0.6543        | 2.51  | 155000 | 0.4693          |
| 0.6596        | 2.53  | 156000 | 0.4511          |
| 1.2077        | 2.55  | 157000 | 0.4506          |
| 0.9068        | 2.56  | 158000 | 0.4580          |
| 0.9328        | 2.58  | 159000 | 0.4602          |
| 0.8111        | 2.6   | 160000 | 0.4607          |
| 0.8047        | 2.61  | 161000 | 0.4475          |
| 0.7448        | 2.63  | 162000 | 0.4532          |
| 1.0685        | 2.64  | 163000 | 0.4628          |
| 0.9309        | 2.66  | 164000 | 0.4569          |
| 0.8024        | 2.68  | 165000 | 0.4571          |
| 0.9119        | 2.69  | 166000 | 0.4572          |
| 0.5736        | 2.71  | 167000 | 0.4567          |
| 0.7446        | 2.73  | 168000 | 0.4481          |
| 1.1721        | 2.74  | 169000 | 0.4449          |
| 0.7552        | 2.76  | 170000 | 0.4429          |
| 0.7927        | 2.77  | 171000 | 0.4358          |
| 1.2197        | 2.79  | 172000 | 0.4301          |
| 0.771         | 2.81  | 173000 | 0.4388          |
| 0.7656        | 2.82  | 174000 | 0.4385          |
| 1.0673        | 2.84  | 175000 | 0.4432          |
| 1.0228        | 2.86  | 176000 | 0.4361          |
| 0.7585        | 2.87  | 177000 | 0.4380          |
| 0.63          | 2.89  | 178000 | 0.4371          |
| 1.0722        | 2.9   | 179000 | 0.4314          |
| 0.7093        | 2.92  | 180000 | 0.4337          |
| 0.468         | 2.94  | 181000 | 0.4341          |
| 0.7841        | 2.95  | 182000 | 0.4313          |
| 0.779         | 2.97  | 183000 | 0.4312          |
| 0.4624        | 2.98  | 184000 | 0.4327          |


### Framework versions

- Transformers 4.33.0.dev0
- Pytorch 2.0.1+cu117
- Datasets 2.13.1
- Tokenizers 0.13.3
