
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# model_mask30p_100epoch

This model is a fine-tuned version of [](https://huggingface.co/) on the None dataset.
It achieves the following results on the evaluation set:
- Loss: 0.2550

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 100

### Training results

| Training Loss | Epoch | Step | Validation Loss |
|:-------------:|:-----:|:----:|:---------------:|
| No log        | 1.0   | 57   | 2.6725          |
| No log        | 2.0   | 114  | 2.6084          |
| No log        | 3.0   | 171  | 2.6114          |
| No log        | 4.0   | 228  | 2.6452          |
| No log        | 5.0   | 285  | 2.6414          |
| No log        | 6.0   | 342  | 2.5859          |
| No log        | 7.0   | 399  | 2.6358          |
| No log        | 8.0   | 456  | 2.6219          |
| 2.6069        | 9.0   | 513  | 2.6005          |
| 2.6069        | 10.0  | 570  | 2.5896          |
| 2.6069        | 11.0  | 627  | 2.5994          |
| 2.6069        | 12.0  | 684  | 2.5500          |
| 2.6069        | 13.0  | 741  | 2.5967          |
| 2.6069        | 14.0  | 798  | 2.5714          |
| 2.6069        | 15.0  | 855  | 2.5651          |
| 2.6069        | 16.0  | 912  | 2.5307          |
| 2.6069        | 17.0  | 969  | 2.3990          |
| 2.5079        | 18.0  | 1026 | 2.3268          |
| 2.5079        | 19.0  | 1083 | 2.3078          |
| 2.5079        | 20.0  | 1140 | 2.3268          |
| 2.5079        | 21.0  | 1197 | 2.3278          |
| 2.5079        | 22.0  | 1254 | 2.3542          |
| 2.5079        | 23.0  | 1311 | 2.3773          |
| 2.5079        | 24.0  | 1368 | 2.3548          |
| 2.5079        | 25.0  | 1425 | 2.3574          |
| 2.5079        | 26.0  | 1482 | 2.3350          |
| 2.2916        | 27.0  | 1539 | 2.2998          |
| 2.2916        | 28.0  | 1596 | 2.3144          |
| 2.2916        | 29.0  | 1653 | 2.3098          |
| 2.2916        | 30.0  | 1710 | 2.3253          |
| 2.2916        | 31.0  | 1767 | 2.3219          |
| 2.2916        | 32.0  | 1824 | 2.2956          |
| 2.2916        | 33.0  | 1881 | 2.3017          |
| 2.2916        | 34.0  | 1938 | 2.3071          |
| 2.2916        | 35.0  | 1995 | 2.3131          |
| 2.2288        | 36.0  | 2052 | 2.3041          |
| 2.2288        | 37.0  | 2109 | 2.2773          |
| 2.2288        | 38.0  | 2166 | 2.2502          |
| 2.2288        | 39.0  | 2223 | 2.2330          |
| 2.2288        | 40.0  | 2280 | 2.2147          |
| 2.2288        | 41.0  | 2337 | 2.1744          |
| 2.2288        | 42.0  | 2394 | 2.1358          |
| 2.2288        | 43.0  | 2451 | 2.1509          |
| 2.1326        | 44.0  | 2508 | 2.1167          |
| 2.1326        | 45.0  | 2565 | 2.0464          |
| 2.1326        | 46.0  | 2622 | 2.0284          |
| 2.1326        | 47.0  | 2679 | 2.0077          |
| 2.1326        | 48.0  | 2736 | 1.9298          |
| 2.1326        | 49.0  | 2793 | 1.8966          |
| 2.1326        | 50.0  | 2850 | 1.8917          |
| 2.1326        | 51.0  | 2907 | 1.7883          |
| 2.1326        | 52.0  | 2964 | 1.6490          |
| 1.8676        | 53.0  | 3021 | 1.6729          |
| 1.8676        | 54.0  | 3078 | 1.5494          |
| 1.8676        | 55.0  | 3135 | 1.5023          |
| 1.8676        | 56.0  | 3192 | 1.3941          |
| 1.8676        | 57.0  | 3249 | 1.3139          |
| 1.8676        | 58.0  | 3306 | 1.2355          |
| 1.8676        | 59.0  | 3363 | 1.1424          |
| 1.8676        | 60.0  | 3420 | 1.0589          |
| 1.8676        | 61.0  | 3477 | 0.9061          |
| 1.3493        | 62.0  | 3534 | 0.8806          |
| 1.3493        | 63.0  | 3591 | 0.7774          |
| 1.3493        | 64.0  | 3648 | 0.6681          |
| 1.3493        | 65.0  | 3705 | 0.6103          |
| 1.3493        | 66.0  | 3762 | 0.5802          |
| 1.3493        | 67.0  | 3819 | 0.4806          |
| 1.3493        | 68.0  | 3876 | 0.4749          |
| 1.3493        | 69.0  | 3933 | 0.4502          |
| 1.3493        | 70.0  | 3990 | 0.4365          |
| 0.7176        | 71.0  | 4047 | 0.3917          |
| 0.7176        | 72.0  | 4104 | 0.3864          |
| 0.7176        | 73.0  | 4161 | 0.3690          |
| 0.7176        | 74.0  | 4218 | 0.3657          |
| 0.7176        | 75.0  | 4275 | 0.3494          |
| 0.7176        | 76.0  | 4332 | 0.3160          |
| 0.7176        | 77.0  | 4389 | 0.3182          |
| 0.7176        | 78.0  | 4446 | 0.2680          |
| 0.4147        | 79.0  | 4503 | 0.3013          |
| 0.4147        | 80.0  | 4560 | 0.2942          |
| 0.4147        | 81.0  | 4617 | 0.2767          |
| 0.4147        | 82.0  | 4674 | 0.3180          |
| 0.4147        | 83.0  | 4731 | 0.2372          |
| 0.4147        | 84.0  | 4788 | 0.2922          |
| 0.4147        | 85.0  | 4845 | 0.2649          |
| 0.4147        | 86.0  | 4902 | 0.2603          |
| 0.4147        | 87.0  | 4959 | 0.2553          |
| 0.3218        | 88.0  | 5016 | 0.2187          |
| 0.3218        | 89.0  | 5073 | 0.2584          |
| 0.3218        | 90.0  | 5130 | 0.2299          |
| 0.3218        | 91.0  | 5187 | 0.2611          |
| 0.3218        | 92.0  | 5244 | 0.2511          |
| 0.3218        | 93.0  | 5301 | 0.2506          |
| 0.3218        | 94.0  | 5358 | 0.2380          |
| 0.3218        | 95.0  | 5415 | 0.2522          |
| 0.3218        | 96.0  | 5472 | 0.2313          |
| 0.2876        | 97.0  | 5529 | 0.2121          |
| 0.2876        | 98.0  | 5586 | 0.2049          |
| 0.2876        | 99.0  | 5643 | 0.1957          |
| 0.2876        | 100.0 | 5700 | 0.2581          |


### Framework versions

- Transformers 4.35.2
- Pytorch 2.1.0+cu118
- Datasets 2.15.0
- Tokenizers 0.15.0

{"tags": ["generated_from_trainer"], "model-index": [{"name": "model_mask30p_100epoch", "results": []}]}
transformers tensorboard safetensors distilbert fill-mask generated_from_trainer autotrain_compatible endpoints_compatible region:us