transformers tensorboard safetensors bert generated_from_trainer base_model:veriga/distilbert-base-uncased-finetuned-cola license:apache-2.0 endpoints_compatible region:us
---
license: apache-2.0
base_model: veriga/distilbert-base-uncased-finetuned-cola
tags:
- generated_from_trainer
metrics:
- accuracy
- f1
model-index:
- name: distilbert-base-uncased-finetuned-cola
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# distilbert-base-uncased-finetuned-cola

This model is a fine-tuned version of [veriga/distilbert-base-uncased-finetuned-cola](https://huggingface.co/veriga/distilbert-base-uncased-finetuned-cola) on an unknown dataset.
It achieves the following results on the evaluation set:
- Loss: 3.8100
- Accuracy: 0.9556
- F1: 0.0017

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 256
- eval_batch_size: 256
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 100

### Training results

| Training Loss | Epoch | Step | Validation Loss | Accuracy | F1     |
|:-------------:|:-----:|:----:|:---------------:|:--------:|:------:|
| No log        | 1.0   | 37   | 3.7942          | 0.9556   | 0.0017 |
| No log        | 2.0   | 74   | 3.7810          | 0.9556   | 0.0    |
| No log        | 3.0   | 111  | 3.8160          | 0.9557   | 0.0026 |
| No log        | 4.0   | 148  | 3.8256          | 0.9557   | 0.0026 |
| No log        | 5.0   | 185  | 3.7971          | 0.9558   | 0.0052 |
| No log        | 6.0   | 222  | 3.8099          | 0.9557   | 0.0026 |
| No log        | 7.0   | 259  | 3.7882          | 0.9556   | 0.0009 |
| No log        | 8.0   | 296  | 3.8088          | 0.9556   | 0.0017 |
| No log        | 9.0   | 333  | 3.8060          | 0.9557   | 0.0026 |
| No log        | 10.0  | 370  | 3.8122          | 0.9557   | 0.0034 |
| No log        | 11.0  | 407  | 3.8023          | 0.9556   | 0.0    |
| No log        | 12.0  | 444  | 3.7889          | 0.9556   | 0.0017 |
| No log        | 13.0  | 481  | 3.8140          | 0.9556   | 0.0    |
| 3.8738        | 14.0  | 518  | 3.8300          | 0.9556   | 0.0009 |
| 3.8738        | 15.0  | 555  | 3.8003          | 0.9556   | 0.0    |
| 3.8738        | 16.0  | 592  | 3.8238          | 0.9556   | 0.0017 |
| 3.8738        | 17.0  | 629  | 3.7945          | 0.9556   | 0.0    |
| 3.8738        | 18.0  | 666  | 3.8228          | 0.9557   | 0.0043 |
| 3.8738        | 19.0  | 703  | 3.8044          | 0.9556   | 0.0    |
| 3.8738        | 20.0  | 740  | 3.8049          | 0.9556   | 0.0017 |
| 3.8738        | 21.0  | 777  | 3.8031          | 0.9556   | 0.0017 |
| 3.8738        | 22.0  | 814  | 3.8092          | 0.9556   | 0.0017 |
| 3.8738        | 23.0  | 851  | 3.8090          | 0.9559   | 0.0086 |
| 3.8738        | 24.0  | 888  | 3.7972          | 0.9556   | 0.0    |
| 3.8738        | 25.0  | 925  | 3.8085          | 0.9556   | 0.0    |
| 3.8738        | 26.0  | 962  | 3.7968          | 0.9557   | 0.0043 |
| 3.8738        | 27.0  | 999  | 3.8240          | 0.9557   | 0.0026 |
| 3.8886        | 28.0  | 1036 | 3.7986          | 0.9556   | 0.0009 |
| 3.8886        | 29.0  | 1073 | 3.7939          | 0.9556   | 0.0009 |
| 3.8886        | 30.0  | 1110 | 3.8051          | 0.9557   | 0.0026 |
| 3.8886        | 31.0  | 1147 | 3.8014          | 0.9556   | 0.0009 |
| 3.8886        | 32.0  | 1184 | 3.8082          | 0.9556   | 0.0017 |
| 3.8886        | 33.0  | 1221 | 3.8032          | 0.9557   | 0.0026 |
| 3.8886        | 34.0  | 1258 | 3.8092          | 0.9557   | 0.0026 |
| 3.8886        | 35.0  | 1295 | 3.7875          | 0.9557   | 0.0026 |
| 3.8886        | 36.0  | 1332 | 3.8107          | 0.9557   | 0.0026 |
| 3.8886        | 37.0  | 1369 | 3.7927          | 0.9556   | 0.0009 |
| 3.8886        | 38.0  | 1406 | 3.7875          | 0.9556   | 0.0017 |
| 3.8886        | 39.0  | 1443 | 3.8008          | 0.9556   | 0.0017 |
| 3.8886        | 40.0  | 1480 | 3.7967          | 0.9556   | 0.0017 |
| 3.8388        | 41.0  | 1517 | 3.8110          | 0.9564   | 0.0197 |
| 3.8388        | 42.0  | 1554 | 3.8134          | 0.9557   | 0.0026 |
| 3.8388        | 43.0  | 1591 | 3.8047          | 0.9556   | 0.0017 |
| 3.8388        | 44.0  | 1628 | 3.8008          | 0.9556   | 0.0017 |
| 3.8388        | 45.0  | 1665 | 3.8170          | 0.9557   | 0.0026 |
| 3.8388        | 46.0  | 1702 | 3.8085          | 0.9556   | 0.0017 |
| 3.8388        | 47.0  | 1739 | 3.8142          | 0.9556   | 0.0017 |
| 3.8388        | 48.0  | 1776 | 3.8048          | 0.9557   | 0.0034 |
| 3.8388        | 49.0  | 1813 | 3.8121          | 0.9557   | 0.0034 |
| 3.8388        | 50.0  | 1850 | 3.7905          | 0.9556   | 0.0    |
| 3.8388        | 51.0  | 1887 | 3.7977          | 0.9556   | 0.0017 |
| 3.8388        | 52.0  | 1924 | 3.8083          | 0.9564   | 0.0197 |
| 3.8388        | 53.0  | 1961 | 3.7955          | 0.9557   | 0.0026 |
| 3.8388        | 54.0  | 1998 | 3.8105          | 0.9564   | 0.0197 |
| 3.858         | 55.0  | 2035 | 3.8086          | 0.9557   | 0.0043 |
| 3.858         | 56.0  | 2072 | 3.8025          | 0.9557   | 0.0034 |
| 3.858         | 57.0  | 2109 | 3.7983          | 0.9556   | 0.0009 |
| 3.858         | 58.0  | 2146 | 3.8122          | 0.9564   | 0.0197 |
| 3.858         | 59.0  | 2183 | 3.8152          | 0.9557   | 0.0043 |
| 3.858         | 60.0  | 2220 | 3.8014          | 0.9556   | 0.0017 |
| 3.858         | 61.0  | 2257 | 3.8000          | 0.9556   | 0.0017 |
| 3.858         | 62.0  | 2294 | 3.8049          | 0.9556   | 0.0017 |
| 3.858         | 63.0  | 2331 | 3.7995          | 0.9556   | 0.0017 |
| 3.858         | 64.0  | 2368 | 3.8088          | 0.9556   | 0.0017 |
| 3.858         | 65.0  | 2405 | 3.8087          | 0.9556   | 0.0009 |
| 3.858         | 66.0  | 2442 | 3.8093          | 0.9557   | 0.0043 |
| 3.858         | 67.0  | 2479 | 3.8257          | 0.9564   | 0.0197 |
| 3.8513        | 68.0  | 2516 | 3.8136          | 0.9556   | 0.0009 |
| 3.8513        | 69.0  | 2553 | 3.8211          | 0.9564   | 0.0197 |
| 3.8513        | 70.0  | 2590 | 3.8055          | 0.9556   | 0.0    |
| 3.8513        | 71.0  | 2627 | 3.8070          | 0.9556   | 0.0    |
| 3.8513        | 72.0  | 2664 | 3.8054          | 0.9556   | 0.0    |
| 3.8513        | 73.0  | 2701 | 3.7996          | 0.9556   | 0.0009 |
| 3.8513        | 74.0  | 2738 | 3.8220          | 0.9564   | 0.0197 |
| 3.8513        | 75.0  | 2775 | 3.8093          | 0.9564   | 0.0197 |
| 3.8513        | 76.0  | 2812 | 3.8100          | 0.9564   | 0.0197 |
| 3.8513        | 77.0  | 2849 | 3.8065          | 0.9564   | 0.0197 |
| 3.8513        | 78.0  | 2886 | 3.8056          | 0.9556   | 0.0017 |
| 3.8513        | 79.0  | 2923 | 3.8060          | 0.9564   | 0.0197 |
| 3.8513        | 80.0  | 2960 | 3.8124          | 0.9564   | 0.0197 |
| 3.8513        | 81.0  | 2997 | 3.8047          | 0.9564   | 0.0197 |
| 3.8615        | 82.0  | 3034 | 3.8096          | 0.9557   | 0.0043 |
| 3.8615        | 83.0  | 3071 | 3.8096          | 0.9556   | 0.0009 |
| 3.8615        | 84.0  | 3108 | 3.8052          | 0.9556   | 0.0009 |
| 3.8615        | 85.0  | 3145 | 3.8035          | 0.9556   | 0.0009 |
| 3.8615        | 86.0  | 3182 | 3.8077          | 0.9556   | 0.0017 |
| 3.8615        | 87.0  | 3219 | 3.8029          | 0.9556   | 0.0017 |
| 3.8615        | 88.0  | 3256 | 3.8057          | 0.9556   | 0.0017 |
| 3.8615        | 89.0  | 3293 | 3.8038          | 0.9556   | 0.0017 |
| 3.8615        | 90.0  | 3330 | 3.8043          | 0.9556   | 0.0017 |
| 3.8615        | 91.0  | 3367 | 3.8036          | 0.9556   | 0.0017 |
| 3.8615        | 92.0  | 3404 | 3.8076          | 0.9556   | 0.0017 |
| 3.8615        | 93.0  | 3441 | 3.8103          | 0.9556   | 0.0017 |
| 3.8615        | 94.0  | 3478 | 3.8104          | 0.9556   | 0.0017 |
| 3.8533        | 95.0  | 3515 | 3.8106          | 0.9556   | 0.0017 |
| 3.8533        | 96.0  | 3552 | 3.8089          | 0.9556   | 0.0017 |
| 3.8533        | 97.0  | 3589 | 3.8098          | 0.9556   | 0.0017 |
| 3.8533        | 98.0  | 3626 | 3.8097          | 0.9556   | 0.0017 |
| 3.8533        | 99.0  | 3663 | 3.8100          | 0.9556   | 0.0017 |
| 3.8533        | 100.0 | 3700 | 3.8100          | 0.9556   | 0.0017 |


### Framework versions

- Transformers 4.36.2
- Pytorch 1.14.0.dev20221204+cu117
- Datasets 2.2.2
- Tokenizers 0.15.0
