
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# sembr2023-distilbert-base-cased

This model is a fine-tuned version of [distilbert-base-cased](https://huggingface.co/distilbert-base-cased) on an unknown dataset.
It achieves the following results on the evaluation set:
- Loss: 0.2214
- Precision: 0.7952
- Recall: 0.8261
- F1: 0.8104
- Iou: 0.6812
- Accuracy: 0.9665
- Balanced Accuracy: 0.9030
- Overall Accuracy: 0.9525

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 0.0001
- train_batch_size: 64
- eval_batch_size: 128
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: cosine
- training_steps: 1000

### Training results

| Training Loss | Epoch | Step | Validation Loss | Precision | Recall | F1     | Iou    | Accuracy | Balanced Accuracy | Overall Accuracy |
|:-------------:|:-----:|:----:|:---------------:|:---------:|:------:|:------:|:------:|:--------:|:-----------------:|:----------------:|
| 0.3717        | 0.06  | 10   | 0.3886          | 0         | 0.0    | 0.0    | 0.0    | 0.9133   | 0.5               | 0.9133           |
| 0.3692        | 0.12  | 20   | 0.3552          | 0         | 0.0    | 0.0    | 0.0    | 0.9133   | 0.5               | 0.9133           |
| 0.2579        | 0.17  | 30   | 0.2734          | 0.8923    | 0.1841 | 0.3052 | 0.1801 | 0.9274   | 0.5910            | 0.9255           |
| 0.2135        | 0.23  | 40   | 0.2224          | 0.7632    | 0.6267 | 0.6882 | 0.5247 | 0.9508   | 0.8041            | 0.9348           |
| 0.225         | 0.29  | 50   | 0.1935          | 0.8426    | 0.6378 | 0.7260 | 0.5699 | 0.9583   | 0.8132            | 0.9427           |
| 0.1637        | 0.35  | 60   | 0.1785          | 0.8115    | 0.6951 | 0.7488 | 0.5985 | 0.9596   | 0.8399            | 0.9470           |
| 0.1497        | 0.4   | 70   | 0.1837          | 0.8159    | 0.6961 | 0.7513 | 0.6016 | 0.9601   | 0.8406            | 0.9434           |
| 0.1248        | 0.46  | 80   | 0.1758          | 0.7920    | 0.7523 | 0.7717 | 0.6282 | 0.9614   | 0.8668            | 0.9446           |
| 0.1297        | 0.52  | 90   | 0.1796          | 0.7740    | 0.7933 | 0.7835 | 0.6441 | 0.9620   | 0.8857            | 0.9430           |
| 0.1321        | 0.58  | 100  | 0.1721          | 0.8616    | 0.7178 | 0.7832 | 0.6436 | 0.9656   | 0.8534            | 0.9496           |
| 0.1058        | 0.64  | 110  | 0.1572          | 0.8132    | 0.7766 | 0.7945 | 0.6591 | 0.9652   | 0.8799            | 0.9494           |
| 0.1183        | 0.69  | 120  | 0.1734          | 0.8084    | 0.7792 | 0.7935 | 0.6578 | 0.9649   | 0.8809            | 0.9470           |
| 0.1195        | 0.75  | 130  | 0.1652          | 0.7753    | 0.7952 | 0.7851 | 0.6462 | 0.9623   | 0.8867            | 0.9463           |
| 0.0996        | 0.81  | 140  | 0.1433          | 0.8292    | 0.7684 | 0.7977 | 0.6634 | 0.9662   | 0.8767            | 0.9527           |
| 0.1009        | 0.87  | 150  | 0.1817          | 0.8181    | 0.7808 | 0.7990 | 0.6653 | 0.9660   | 0.8822            | 0.9480           |
| 0.0953        | 0.92  | 160  | 0.1554          | 0.8669    | 0.7245 | 0.7893 | 0.6519 | 0.9665   | 0.8570            | 0.9524           |
| 0.1077        | 0.98  | 170  | 0.1556          | 0.8261    | 0.7752 | 0.7998 | 0.6664 | 0.9664   | 0.8798            | 0.9512           |
| 0.0981        | 1.04  | 180  | 0.1526          | 0.8283    | 0.7703 | 0.7982 | 0.6642 | 0.9663   | 0.8776            | 0.9520           |
| 0.0982        | 1.1   | 190  | 0.1547          | 0.8001    | 0.7976 | 0.7989 | 0.6651 | 0.9652   | 0.8894            | 0.9504           |
| 0.0789        | 1.16  | 200  | 0.1606          | 0.8135    | 0.7947 | 0.8040 | 0.6722 | 0.9664   | 0.8887            | 0.9513           |
| 0.0829        | 1.21  | 210  | 0.1566          | 0.8244    | 0.7872 | 0.8054 | 0.6741 | 0.9670   | 0.8856            | 0.9517           |
| 0.0742        | 1.27  | 220  | 0.1680          | 0.8167    | 0.7895 | 0.8029 | 0.6707 | 0.9664   | 0.8864            | 0.9506           |
| 0.084         | 1.33  | 230  | 0.1680          | 0.8197    | 0.7824 | 0.8006 | 0.6675 | 0.9662   | 0.8830            | 0.9511           |
| 0.0702        | 1.39  | 240  | 0.1653          | 0.8184    | 0.7996 | 0.8089 | 0.6791 | 0.9673   | 0.8914            | 0.9510           |
| 0.0713        | 1.45  | 250  | 0.1675          | 0.7844    | 0.8184 | 0.8010 | 0.6681 | 0.9648   | 0.8985            | 0.9492           |
| 0.0763        | 1.5   | 260  | 0.1501          | 0.8239    | 0.7833 | 0.8031 | 0.6710 | 0.9667   | 0.8837            | 0.9532           |
| 0.0738        | 1.56  | 270  | 0.1518          | 0.8203    | 0.7962 | 0.8081 | 0.6780 | 0.9672   | 0.8898            | 0.9527           |
| 0.0736        | 1.62  | 280  | 0.1624          | 0.7849    | 0.8222 | 0.8031 | 0.6710 | 0.9651   | 0.9004            | 0.9508           |
| 0.0659        | 1.68  | 290  | 0.1735          | 0.7775    | 0.8308 | 0.8033 | 0.6712 | 0.9647   | 0.9041            | 0.9496           |
| 0.0653        | 1.73  | 300  | 0.1586          | 0.7828    | 0.8224 | 0.8022 | 0.6697 | 0.9648   | 0.9004            | 0.9503           |
| 0.0635        | 1.79  | 310  | 0.1720          | 0.8091    | 0.8033 | 0.8062 | 0.6753 | 0.9665   | 0.8927            | 0.9510           |
| 0.0724        | 1.85  | 320  | 0.1588          | 0.8057    | 0.8033 | 0.8045 | 0.6729 | 0.9662   | 0.8925            | 0.9531           |
| 0.0612        | 1.91  | 330  | 0.1818          | 0.7828    | 0.8222 | 0.8020 | 0.6695 | 0.9648   | 0.9003            | 0.9488           |
| 0.0612        | 1.97  | 340  | 0.1704          | 0.8235    | 0.7893 | 0.8060 | 0.6751 | 0.9671   | 0.8866            | 0.9526           |
| 0.0592        | 2.02  | 350  | 0.1634          | 0.8002    | 0.7929 | 0.7965 | 0.6618 | 0.9649   | 0.8870            | 0.9520           |
| 0.0474        | 2.08  | 360  | 0.1835          | 0.7931    | 0.8120 | 0.8025 | 0.6701 | 0.9654   | 0.8960            | 0.9506           |
| 0.0484        | 2.14  | 370  | 0.1790          | 0.8123    | 0.7941 | 0.8031 | 0.6710 | 0.9663   | 0.8883            | 0.9522           |
| 0.0524        | 2.2   | 380  | 0.1812          | 0.7702    | 0.8291 | 0.7985 | 0.6646 | 0.9637   | 0.9028            | 0.9499           |
| 0.052         | 2.25  | 390  | 0.1716          | 0.8041    | 0.7964 | 0.8002 | 0.6670 | 0.9655   | 0.8890            | 0.9533           |
| 0.0443        | 2.31  | 400  | 0.1676          | 0.8054    | 0.7976 | 0.8015 | 0.6687 | 0.9658   | 0.8897            | 0.9535           |
| 0.057         | 2.37  | 410  | 0.1836          | 0.8028    | 0.8084 | 0.8056 | 0.6745 | 0.9662   | 0.8948            | 0.9507           |
| 0.0414        | 2.43  | 420  | 0.1791          | 0.8049    | 0.8053 | 0.8051 | 0.6737 | 0.9662   | 0.8934            | 0.9527           |
| 0.0471        | 2.49  | 430  | 0.1771          | 0.7964    | 0.8126 | 0.8044 | 0.6728 | 0.9658   | 0.8965            | 0.9527           |
| 0.039         | 2.54  | 440  | 0.1773          | 0.8066    | 0.8021 | 0.8043 | 0.6727 | 0.9662   | 0.8919            | 0.9537           |
| 0.0543        | 2.6   | 450  | 0.1855          | 0.7887    | 0.8193 | 0.8037 | 0.6718 | 0.9653   | 0.8992            | 0.9511           |
| 0.0398        | 2.66  | 460  | 0.1959          | 0.7938    | 0.8147 | 0.8041 | 0.6724 | 0.9656   | 0.8973            | 0.9504           |
| 0.0419        | 2.72  | 470  | 0.1944          | 0.7847    | 0.8286 | 0.8060 | 0.6751 | 0.9654   | 0.9035            | 0.9498           |
| 0.0436        | 2.77  | 480  | 0.1869          | 0.8002    | 0.8109 | 0.8055 | 0.6744 | 0.9661   | 0.8958            | 0.9520           |
| 0.0497        | 2.83  | 490  | 0.1850          | 0.7736    | 0.8422 | 0.8065 | 0.6757 | 0.9650   | 0.9094            | 0.9501           |
| 0.0408        | 2.89  | 500  | 0.1883          | 0.8178    | 0.7962 | 0.8068 | 0.6762 | 0.9670   | 0.8897            | 0.9527           |
| 0.0332        | 2.95  | 510  | 0.1883          | 0.7913    | 0.8188 | 0.8048 | 0.6733 | 0.9656   | 0.8991            | 0.9516           |
| 0.0382        | 3.01  | 520  | 0.2008          | 0.7914    | 0.8307 | 0.8106 | 0.6815 | 0.9664   | 0.9049            | 0.9515           |
| 0.047         | 3.06  | 530  | 0.1913          | 0.8137    | 0.8013 | 0.8075 | 0.6771 | 0.9669   | 0.8920            | 0.9522           |
| 0.0327        | 3.12  | 540  | 0.1969          | 0.7993    | 0.8168 | 0.8080 | 0.6778 | 0.9664   | 0.8987            | 0.9518           |
| 0.0338        | 3.18  | 550  | 0.1989          | 0.7962    | 0.8173 | 0.8066 | 0.6759 | 0.9660   | 0.8987            | 0.9518           |
| 0.0332        | 3.24  | 560  | 0.2004          | 0.7999    | 0.8178 | 0.8087 | 0.6789 | 0.9665   | 0.8992            | 0.9518           |
| 0.0308        | 3.29  | 570  | 0.1964          | 0.8126    | 0.8092 | 0.8109 | 0.6819 | 0.9673   | 0.8957            | 0.9537           |
| 0.0348        | 3.35  | 580  | 0.2032          | 0.7902    | 0.8239 | 0.8067 | 0.6761 | 0.9658   | 0.9016            | 0.9515           |
| 0.0351        | 3.41  | 590  | 0.2064          | 0.7855    | 0.8218 | 0.8032 | 0.6712 | 0.9651   | 0.9003            | 0.9511           |
| 0.0301        | 3.47  | 600  | 0.2118          | 0.7872    | 0.8265 | 0.8063 | 0.6755 | 0.9656   | 0.9026            | 0.9505           |
| 0.0261        | 3.53  | 610  | 0.1997          | 0.7991    | 0.8194 | 0.8091 | 0.6794 | 0.9665   | 0.8999            | 0.9522           |
| 0.0282        | 3.58  | 620  | 0.1950          | 0.8029    | 0.8114 | 0.8071 | 0.6766 | 0.9664   | 0.8962            | 0.9527           |
| 0.0326        | 3.64  | 630  | 0.2038          | 0.7873    | 0.8290 | 0.8076 | 0.6773 | 0.9658   | 0.9039            | 0.9516           |
| 0.0353        | 3.7   | 640  | 0.2010          | 0.7930    | 0.8228 | 0.8076 | 0.6773 | 0.9660   | 0.9012            | 0.9514           |
| 0.0348        | 3.76  | 650  | 0.2043          | 0.7949    | 0.8243 | 0.8093 | 0.6797 | 0.9663   | 0.9021            | 0.9519           |
| 0.0296        | 3.82  | 660  | 0.2050          | 0.7976    | 0.8226 | 0.8099 | 0.6805 | 0.9665   | 0.9014            | 0.9529           |
| 0.0287        | 3.87  | 670  | 0.2158          | 0.7820    | 0.8318 | 0.8061 | 0.6752 | 0.9653   | 0.9049            | 0.9504           |
| 0.024         | 3.93  | 680  | 0.2110          | 0.7847    | 0.8294 | 0.8065 | 0.6757 | 0.9655   | 0.9039            | 0.9512           |
| 0.0274        | 3.99  | 690  | 0.2075          | 0.7937    | 0.8254 | 0.8092 | 0.6796 | 0.9663   | 0.9025            | 0.9523           |
| 0.0247        | 4.05  | 700  | 0.2130          | 0.7995    | 0.8210 | 0.8101 | 0.6808 | 0.9666   | 0.9007            | 0.9525           |
| 0.0202        | 4.1   | 710  | 0.2142          | 0.7955    | 0.8215 | 0.8083 | 0.6782 | 0.9662   | 0.9007            | 0.9518           |
| 0.0245        | 4.16  | 720  | 0.2120          | 0.7965    | 0.8195 | 0.8078 | 0.6776 | 0.9662   | 0.8998            | 0.9516           |
| 0.0214        | 4.22  | 730  | 0.2151          | 0.7899    | 0.8256 | 0.8074 | 0.6770 | 0.9659   | 0.9024            | 0.9515           |
| 0.0202        | 4.28  | 740  | 0.2145          | 0.7963    | 0.8220 | 0.8089 | 0.6792 | 0.9664   | 0.9010            | 0.9520           |
| 0.0257        | 4.34  | 750  | 0.2181          | 0.7960    | 0.8217 | 0.8087 | 0.6788 | 0.9663   | 0.9009            | 0.9520           |
| 0.0271        | 4.39  | 760  | 0.2151          | 0.7953    | 0.8232 | 0.8090 | 0.6793 | 0.9663   | 0.9015            | 0.9518           |
| 0.0279        | 4.45  | 770  | 0.2196          | 0.7955    | 0.8237 | 0.8094 | 0.6798 | 0.9664   | 0.9018            | 0.9521           |
| 0.0273        | 4.51  | 780  | 0.2194          | 0.7984    | 0.8256 | 0.8118 | 0.6832 | 0.9668   | 0.9029            | 0.9523           |
| 0.018         | 4.57  | 790  | 0.2201          | 0.7985    | 0.8247 | 0.8114 | 0.6826 | 0.9668   | 0.9025            | 0.9526           |
| 0.0275        | 4.62  | 800  | 0.2204          | 0.7893    | 0.8358 | 0.8119 | 0.6834 | 0.9664   | 0.9073            | 0.9519           |
| 0.0198        | 4.68  | 810  | 0.2160          | 0.7983    | 0.8232 | 0.8105 | 0.6814 | 0.9666   | 0.9017            | 0.9526           |
| 0.019         | 4.74  | 820  | 0.2109          | 0.7961    | 0.8243 | 0.8100 | 0.6806 | 0.9665   | 0.9021            | 0.9527           |
| 0.0236        | 4.8   | 830  | 0.2208          | 0.7956    | 0.8238 | 0.8094 | 0.6799 | 0.9664   | 0.9019            | 0.9521           |
| 0.0177        | 4.86  | 840  | 0.2217          | 0.7900    | 0.8301 | 0.8095 | 0.6800 | 0.9661   | 0.9046            | 0.9519           |
| 0.0209        | 4.91  | 850  | 0.2226          | 0.7927    | 0.8285 | 0.8102 | 0.6810 | 0.9664   | 0.9040            | 0.9522           |
| 0.0241        | 4.97  | 860  | 0.2215          | 0.7915    | 0.8276 | 0.8091 | 0.6794 | 0.9662   | 0.9035            | 0.9521           |
| 0.0211        | 5.03  | 870  | 0.2181          | 0.7957    | 0.8242 | 0.8097 | 0.6802 | 0.9664   | 0.9020            | 0.9525           |
| 0.0234        | 5.09  | 880  | 0.2171          | 0.7975    | 0.8224 | 0.8098 | 0.6803 | 0.9665   | 0.9013            | 0.9526           |
| 0.0201        | 5.14  | 890  | 0.2191          | 0.7925    | 0.8265 | 0.8092 | 0.6795 | 0.9662   | 0.9030            | 0.9523           |
| 0.0211        | 5.2   | 900  | 0.2175          | 0.7957    | 0.8238 | 0.8095 | 0.6799 | 0.9664   | 0.9019            | 0.9526           |
| 0.0234        | 5.26  | 910  | 0.2207          | 0.7913    | 0.8291 | 0.8097 | 0.6803 | 0.9662   | 0.9042            | 0.9522           |
| 0.023         | 5.32  | 920  | 0.2202          | 0.7965    | 0.8234 | 0.8098 | 0.6803 | 0.9665   | 0.9017            | 0.9524           |
| 0.0192        | 5.38  | 930  | 0.2203          | 0.7969    | 0.8239 | 0.8102 | 0.6809 | 0.9665   | 0.9020            | 0.9525           |
| 0.0217        | 5.43  | 940  | 0.2206          | 0.7956    | 0.8255 | 0.8103 | 0.6811 | 0.9665   | 0.9027            | 0.9524           |
| 0.0195        | 5.49  | 950  | 0.2213          | 0.7953    | 0.8259 | 0.8103 | 0.6811 | 0.9665   | 0.9029            | 0.9524           |
| 0.0285        | 5.55  | 960  | 0.2214          | 0.7955    | 0.8254 | 0.8102 | 0.6809 | 0.9665   | 0.9026            | 0.9524           |
| 0.0263        | 5.61  | 970  | 0.2213          | 0.7955    | 0.8254 | 0.8102 | 0.6809 | 0.9665   | 0.9026            | 0.9524           |
| 0.02          | 5.66  | 980  | 0.2214          | 0.7951    | 0.8258 | 0.8101 | 0.6809 | 0.9665   | 0.9028            | 0.9524           |
| 0.021         | 5.72  | 990  | 0.2214          | 0.7952    | 0.8261 | 0.8104 | 0.6812 | 0.9665   | 0.9030            | 0.9525           |
| 0.0233        | 5.78  | 1000 | 0.2214          | 0.7952    | 0.8261 | 0.8104 | 0.6812 | 0.9665   | 0.9030            | 0.9525           |


### Framework versions

- Transformers 4.34.1
- Pytorch 2.0.1
- Datasets 2.14.6
- Tokenizers 0.14.1

{"license": "apache-2.0", "tags": ["generated_from_trainer"], "metrics": ["precision", "recall", "f1", "accuracy"], "base_model": "distilbert-base-cased", "model-index": [{"name": "sembr2023-distilbert-base-cased", "results": []}]}
transformers pytorch distilbert token-classification generated_from_trainer base_model:distilbert-base-cased license:apache-2.0 autotrain_compatible endpoints_compatible region:us