peft tensorboard safetensors generated_from_trainer base_model:meta-llama/Llama-2-7b-hf region:us
---
library_name: peft
tags:
- generated_from_trainer
base_model: meta-llama/Llama-2-7b-hf
model-index:
- name: ice_llama2_v1
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# ice_llama2_v1

This model is a fine-tuned version of [meta-llama/Llama-2-7b-hf](https://huggingface.co/meta-llama/Llama-2-7b-hf) on an unknown dataset.
It achieves the following results on the evaluation set:
- Loss: 1.9931

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: cosine
- num_epochs: 3

### Training results

| Training Loss | Epoch | Step   | Validation Loss |
|:-------------:|:-----:|:------:|:---------------:|
| 2.3158        | 0.03  | 10000  | 2.2641          |
| 2.2183        | 0.07  | 20000  | 2.1728          |
| 2.2155        | 0.1   | 30000  | 2.1331          |
| 2.1487        | 0.14  | 40000  | 2.1068          |
| 2.1569        | 0.17  | 50000  | 2.0883          |
| 2.15          | 0.2   | 60000  | 2.0753          |
| 2.1516        | 0.24  | 70000  | 2.0657          |
| 2.0938        | 0.27  | 80000  | 2.0576          |
| 2.1507        | 0.3   | 90000  | 2.0500          |
| 2.1218        | 0.34  | 100000 | 2.0449          |
| 2.1107        | 0.37  | 110000 | 2.0405          |
| 2.1035        | 0.41  | 120000 | 2.0354          |
| 2.0684        | 0.44  | 130000 | 2.0327          |
| 2.1331        | 0.47  | 140000 | 2.0288          |
| 2.0844        | 0.51  | 150000 | 2.0269          |
| 2.1046        | 0.54  | 160000 | 2.0239          |
| 2.0912        | 0.57  | 170000 | 2.0208          |
| 2.0761        | 0.61  | 180000 | 2.0196          |
| 2.0917        | 0.64  | 190000 | 2.0171          |
| 2.1177        | 0.68  | 200000 | 2.0157          |
| 2.0647        | 0.71  | 210000 | 2.0134          |
| 2.0823        | 0.74  | 220000 | 2.0121          |
| 2.0761        | 0.78  | 230000 | 2.0105          |
| 2.0729        | 0.81  | 240000 | 2.0094          |
| 2.0495        | 0.85  | 250000 | 2.0086          |
| 2.0507        | 0.88  | 260000 | 2.0069          |
| 2.0933        | 0.91  | 270000 | 2.0058          |
| 2.1401        | 0.95  | 280000 | 2.0040          |
| 2.0917        | 0.98  | 290000 | 2.0034          |
| 2.0667        | 1.01  | 300000 | 2.0029          |
| 2.0958        | 1.05  | 310000 | 2.0017          |
| 2.071         | 1.08  | 320000 | 2.0012          |
| 2.0867        | 1.12  | 330000 | 2.0004          |
| 2.0705        | 1.15  | 340000 | 1.9993          |
| 2.0702        | 1.18  | 350000 | 1.9990          |
| 2.081         | 1.22  | 360000 | 1.9990          |
| 2.0671        | 1.25  | 370000 | 1.9980          |
| 2.0735        | 1.28  | 380000 | 1.9977          |
| 2.0681        | 1.32  | 390000 | 1.9969          |
| 2.0642        | 1.35  | 400000 | 1.9966          |
| 2.0773        | 1.39  | 410000 | 1.9962          |
| 2.0999        | 1.42  | 420000 | 1.9959          |
| 2.0637        | 1.45  | 430000 | 1.9958          |
| 2.0986        | 1.49  | 440000 | 1.9952          |
| 2.0821        | 1.52  | 450000 | 1.9953          |
| 2.0524        | 1.56  | 460000 | 1.9950          |
| 2.0594        | 1.59  | 470000 | 1.9945          |
| 2.041         | 1.62  | 480000 | 1.9943          |
| 2.0659        | 1.66  | 490000 | 1.9945          |
| 2.0748        | 1.69  | 500000 | 1.9942          |
| 2.0253        | 1.72  | 510000 | 1.9941          |
| 2.0728        | 1.76  | 520000 | 1.9938          |
| 2.0665        | 1.79  | 530000 | 1.9937          |
| 2.0527        | 1.83  | 540000 | 1.9936          |
| 2.0597        | 1.86  | 550000 | 1.9936          |
| 2.054         | 1.89  | 560000 | 1.9935          |
| 2.1144        | 1.93  | 570000 | 1.9935          |
| 2.0751        | 1.96  | 580000 | 1.9934          |
| 2.0791        | 1.99  | 590000 | 1.9934          |
| 2.0473        | 2.03  | 600000 | 1.9933          |
| 2.0804        | 2.06  | 610000 | 1.9933          |
| 2.1045        | 2.1   | 620000 | 1.9932          |
| 2.0622        | 2.13  | 630000 | 1.9933          |
| 2.0323        | 2.16  | 640000 | 1.9932          |
| 2.0698        | 2.2   | 650000 | 1.9932          |
| 2.0932        | 2.23  | 660000 | 1.9931          |
| 2.0854        | 2.27  | 670000 | 1.9932          |
| 2.0592        | 2.3   | 680000 | 1.9931          |
| 2.0576        | 2.33  | 690000 | 1.9931          |
| 2.0627        | 2.37  | 700000 | 1.9932          |
| 2.0731        | 2.4   | 710000 | 1.9931          |
| 2.0547        | 2.43  | 720000 | 1.9931          |
| 2.0534        | 2.47  | 730000 | 1.9932          |
| 2.0552        | 2.5   | 740000 | 1.9931          |
| 2.0583        | 2.54  | 750000 | 1.9931          |
| 2.0702        | 2.57  | 760000 | 1.9931          |
| 2.0673        | 2.6   | 770000 | 1.9932          |
| 2.0382        | 2.64  | 780000 | 1.9932          |
| 2.0758        | 2.67  | 790000 | 1.9932          |
| 2.0885        | 2.7   | 800000 | 1.9931          |
| 2.0663        | 2.74  | 810000 | 1.9931          |
| 2.0677        | 2.77  | 820000 | 1.9931          |
| 2.0502        | 2.81  | 830000 | 1.9931          |
| 2.1153        | 2.84  | 840000 | 1.9931          |
| 2.0621        | 2.87  | 850000 | 1.9931          |
| 2.099         | 2.91  | 860000 | 1.9931          |
| 2.0285        | 2.94  | 870000 | 1.9930          |
| 2.0343        | 2.98  | 880000 | 1.9931          |


### Framework versions

- PEFT 0.8.2
- Transformers 4.38.1
- Pytorch 2.2.0+cu118
- Datasets 2.17.1
- Tokenizers 0.15.2