peft safetensors generated_from_trainer base_model:microsoft/phi-2 license:mit region:us
---
license: mit
library_name: peft
tags:
- generated_from_trainer
base_model: microsoft/phi-2
model-index:
- name: phi-2-hummanize1
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# phi-2-hummanize1

This model is a fine-tuned version of [microsoft/phi-2](https://huggingface.co/microsoft/phi-2) on the None dataset.
It achieves the following results on the evaluation set:
- Loss: 6.7074

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 0.0025
- train_batch_size: 20
- eval_batch_size: 8
- seed: 42
- gradient_accumulation_steps: 20
- total_train_batch_size: 400
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 2
- num_epochs: 2

### Training results

| Training Loss | Epoch | Step | Validation Loss |
|:-------------:|:-----:|:----:|:---------------:|
| No log        | 0.02  | 1    | 2.0864          |
| No log        | 0.05  | 2    | 4.0671          |
| No log        | 0.07  | 3    | 3.6332          |
| No log        | 0.09  | 4    | 2.5537          |
| 3.0197        | 0.11  | 5    | 2.3394          |
| 3.0197        | 0.14  | 6    | 2.8862          |
| 3.0197        | 0.16  | 7    | 2.5140          |
| 3.0197        | 0.18  | 8    | 2.4603          |
| 3.0197        | 0.21  | 9    | 2.2094          |
| 2.5958        | 0.23  | 10   | 2.1767          |
| 2.5958        | 0.25  | 11   | 2.3343          |
| 2.5958        | 0.28  | 12   | 2.2511          |
| 2.5958        | 0.3   | 13   | 2.1854          |
| 2.5958        | 0.32  | 14   | 2.1385          |
| 2.2944        | 0.34  | 15   | 2.3556          |
| 2.2944        | 0.37  | 16   | 2.2056          |
| 2.2944        | 0.39  | 17   | 2.2127          |
| 2.2944        | 0.41  | 18   | 2.1507          |
| 2.2944        | 0.44  | 19   | 2.1388          |
| 2.2841        | 0.46  | 20   | 2.6540          |
| 2.2841        | 0.48  | 21   | 2.8934          |
| 2.2841        | 0.51  | 22   | 3.0981          |
| 2.2841        | 0.53  | 23   | 2.4155          |
| 2.2841        | 0.55  | 24   | 2.1754          |
| 2.7585        | 0.57  | 25   | 2.0927          |
| 2.7585        | 0.6   | 26   | 2.0865          |
| 2.7585        | 0.62  | 27   | 2.2345          |
| 2.7585        | 0.64  | 28   | 2.4123          |
| 2.7585        | 0.67  | 29   | 2.7718          |
| 2.3906        | 0.69  | 30   | 4.2964          |
| 2.3906        | 0.71  | 31   | 6.5295          |
| 2.3906        | 0.73  | 32   | 5.8489          |
| 2.3906        | 0.76  | 33   | 7.2467          |
| 2.3906        | 0.78  | 34   | 7.6353          |
| 6.5839        | 0.8   | 35   | 7.7842          |
| 6.5839        | 0.83  | 36   | 8.8627          |
| 6.5839        | 0.85  | 37   | 7.9511          |
| 6.5839        | 0.87  | 38   | 9.7736          |
| 6.5839        | 0.9   | 39   | 8.3666          |
| 8.6795        | 0.92  | 40   | 8.9768          |
| 8.6795        | 0.94  | 41   | 9.0808          |
| 8.6795        | 0.96  | 42   | 8.5933          |
| 8.6795        | 0.99  | 43   | 8.9317          |
| 8.6795        | 1.01  | 44   | 8.5291          |
| 8.8177        | 1.03  | 45   | 8.5935          |
| 8.8177        | 1.06  | 46   | 8.6773          |
| 8.8177        | 1.08  | 47   | 8.5914          |
| 8.8177        | 1.1   | 48   | 8.5006          |
| 8.8177        | 1.13  | 49   | 8.3959          |
| 8.6883        | 1.15  | 50   | 8.2375          |
| 8.6883        | 1.17  | 51   | 8.2022          |
| 8.6883        | 1.19  | 52   | 8.2063          |
| 8.6883        | 1.22  | 53   | 8.2254          |
| 8.6883        | 1.24  | 54   | 8.3408          |
| 8.4216        | 1.26  | 55   | 8.0367          |
| 8.4216        | 1.29  | 56   | 7.8776          |
| 8.4216        | 1.31  | 57   | 7.6720          |
| 8.4216        | 1.33  | 58   | 7.5050          |
| 8.4216        | 1.35  | 59   | 7.3863          |
| 7.8151        | 1.38  | 60   | 7.3775          |
| 7.8151        | 1.4   | 61   | 7.3820          |
| 7.8151        | 1.42  | 62   | 7.2597          |
| 7.8151        | 1.45  | 63   | 7.1959          |
| 7.8151        | 1.47  | 64   | 7.1233          |
| 7.3639        | 1.49  | 65   | 7.0625          |
| 7.3639        | 1.52  | 66   | 7.0302          |
| 7.3639        | 1.54  | 67   | 6.9862          |
| 7.3639        | 1.56  | 68   | 6.9601          |
| 7.3639        | 1.58  | 69   | 6.9606          |
| 7.1152        | 1.61  | 70   | 6.8977          |
| 7.1152        | 1.63  | 71   | 6.8981          |
| 7.1152        | 1.65  | 72   | 6.8453          |
| 7.1152        | 1.68  | 73   | 6.8523          |
| 7.1152        | 1.7   | 74   | 6.8641          |
| 6.9712        | 1.72  | 75   | 6.8261          |
| 6.9712        | 1.75  | 76   | 6.8273          |
| 6.9712        | 1.77  | 77   | 6.8053          |
| 6.9712        | 1.79  | 78   | 6.7712          |
| 6.9712        | 1.81  | 79   | 6.7542          |
| 6.8925        | 1.84  | 80   | 6.7466          |
| 6.8925        | 1.86  | 81   | 6.7341          |
| 6.8925        | 1.88  | 82   | 6.7255          |
| 6.8925        | 1.91  | 83   | 6.7211          |
| 6.8925        | 1.93  | 84   | 6.7154          |
| 6.8192        | 1.95  | 85   | 6.7103          |
| 6.8192        | 1.97  | 86   | 6.7074          |


### Framework versions

- PEFT 0.8.2
- Transformers 4.38.0.dev0
- Pytorch 2.1.0+cu118
- Datasets 2.17.0
- Tokenizers 0.15.2