transformers pytorch distilbert token-classification generated_from_trainer base_model:distilbert-base-multilingual-cased license:apache-2.0 autotrain_compatible endpoints_compatible region:us
---
license: apache-2.0
base_model: distilbert-base-multilingual-cased
tags:
- generated_from_trainer
metrics:
- precision
- recall
- f1
- accuracy
model-index:
- name: BERT_B01
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# BERT_B01

This model is a fine-tuned version of [distilbert-base-multilingual-cased](https://huggingface.co/distilbert-base-multilingual-cased) on the None dataset.
It achieves the following results on the evaluation set:
- Loss: 0.6902
- Precision: 0.6636
- Recall: 0.6946
- F1: 0.6788
- Accuracy: 0.8776

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 16
- eval_batch_size: 16
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 30

### Training results

| Training Loss | Epoch | Step | Validation Loss | Precision | Recall | F1     | Accuracy |
|:-------------:|:-----:|:----:|:---------------:|:---------:|:------:|:------:|:--------:|
| 1.0749        | 1.0   | 47   | 0.9390          | 0.4203    | 0.3480 | 0.3807 | 0.7831   |
| 0.6411        | 2.0   | 94   | 0.6110          | 0.5948    | 0.5392 | 0.5657 | 0.8452   |
| 0.4786        | 3.0   | 141  | 0.5279          | 0.6784    | 0.6121 | 0.6435 | 0.8630   |
| 0.3573        | 4.0   | 188  | 0.4972          | 0.6462    | 0.6382 | 0.6422 | 0.8691   |
| 0.2824        | 5.0   | 235  | 0.4868          | 0.6339    | 0.6479 | 0.6408 | 0.8689   |
| 0.2434        | 6.0   | 282  | 0.4970          | 0.6490    | 0.6561 | 0.6525 | 0.8715   |
| 0.1854        | 7.0   | 329  | 0.5004          | 0.6578    | 0.6795 | 0.6685 | 0.8721   |
| 0.1336        | 8.0   | 376  | 0.5091          | 0.6508    | 0.6768 | 0.6635 | 0.8736   |
| 0.1186        | 9.0   | 423  | 0.5437          | 0.6340    | 0.6768 | 0.6547 | 0.8739   |
| 0.103         | 10.0  | 470  | 0.5482          | 0.6570    | 0.6823 | 0.6694 | 0.8771   |
| 0.0799        | 11.0  | 517  | 0.5620          | 0.6444    | 0.6781 | 0.6609 | 0.8752   |
| 0.1045        | 12.0  | 564  | 0.5812          | 0.6557    | 0.6864 | 0.6707 | 0.8760   |
| 0.0562        | 13.0  | 611  | 0.6009          | 0.6667    | 0.6850 | 0.6757 | 0.8780   |
| 0.0637        | 14.0  | 658  | 0.5937          | 0.6707    | 0.6946 | 0.6824 | 0.8780   |
| 0.0657        | 15.0  | 705  | 0.6017          | 0.6788    | 0.6946 | 0.6866 | 0.8789   |
| 0.0371        | 16.0  | 752  | 0.6227          | 0.6858    | 0.6905 | 0.6881 | 0.8776   |
| 0.0389        | 17.0  | 799  | 0.6476          | 0.6499    | 0.6919 | 0.6702 | 0.8767   |
| 0.0461        | 18.0  | 846  | 0.6667          | 0.6556    | 0.7043 | 0.6790 | 0.8786   |
| 0.0377        | 19.0  | 893  | 0.6515          | 0.6788    | 0.6919 | 0.6853 | 0.8793   |
| 0.0364        | 20.0  | 940  | 0.6480          | 0.6791    | 0.7015 | 0.6901 | 0.8784   |
| 0.0383        | 21.0  | 987  | 0.6646          | 0.6719    | 0.7070 | 0.6890 | 0.8802   |
| 0.0173        | 22.0  | 1034 | 0.6724          | 0.6750    | 0.7029 | 0.6887 | 0.8793   |
| 0.0613        | 23.0  | 1081 | 0.6779          | 0.6580    | 0.6988 | 0.6778 | 0.8778   |
| 0.0578        | 24.0  | 1128 | 0.6847          | 0.6592    | 0.6864 | 0.6725 | 0.8767   |
| 0.0201        | 25.0  | 1175 | 0.6714          | 0.6706    | 0.7001 | 0.6851 | 0.8791   |
| 0.022         | 26.0  | 1222 | 0.6874          | 0.6667    | 0.6878 | 0.6770 | 0.8782   |
| 0.0298        | 27.0  | 1269 | 0.6926          | 0.6675    | 0.6960 | 0.6815 | 0.8789   |
| 0.03          | 28.0  | 1316 | 0.6895          | 0.6662    | 0.6974 | 0.6815 | 0.8784   |
| 0.0216        | 29.0  | 1363 | 0.6888          | 0.6636    | 0.6946 | 0.6788 | 0.8780   |
| 0.0236        | 30.0  | 1410 | 0.6902          | 0.6636    | 0.6946 | 0.6788 | 0.8776   |


### Framework versions

- Transformers 4.33.3
- Pytorch 2.0.1+cu117
- Datasets 2.14.4
- Tokenizers 0.13.3
