tensorboard generated_from_trainer base_model:bert-base-multilingual-cased license:apache-2.0 region:us
---
license: apache-2.0
base_model: bert-base-multilingual-cased
tags:
- generated_from_trainer
metrics:
- accuracy
model-index:
- name: ms
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# ms

This model is a fine-tuned version of [bert-base-multilingual-cased](https://huggingface.co/bert-base-multilingual-cased) on an unknown dataset.
It achieves the following results on the evaluation set:
- Loss: 1.4335
- Accuracy: 0.7183

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 5e-05
- train_batch_size: 16
- eval_batch_size: 16
- seed: 42
- distributed_type: multi-GPU
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- training_steps: 50000

### Training results

| Training Loss | Epoch | Step  | Validation Loss | Accuracy |
|:-------------:|:-----:|:-----:|:---------------:|:--------:|
| 1.8607        | 0.05  | 500   | 1.6380          | 0.6876   |
| 1.7966        | 0.1   | 1000  | 1.6063          | 0.6916   |
| 1.7581        | 0.15  | 1500  | 1.5902          | 0.6949   |
| 1.7313        | 0.2   | 2000  | 1.5726          | 0.6975   |
| 1.7143        | 0.25  | 2500  | 1.5630          | 0.6992   |
| 1.7074        | 0.3   | 3000  | 1.5562          | 0.6997   |
| 1.699         | 0.34  | 3500  | 1.5440          | 0.7020   |
| 1.6797        | 0.39  | 4000  | 1.5376          | 0.7026   |
| 1.6813        | 0.44  | 4500  | 1.5355          | 0.7028   |
| 1.6641        | 0.49  | 5000  | 1.5276          | 0.7041   |
| 1.672         | 0.54  | 5500  | 1.5212          | 0.7048   |
| 1.6506        | 0.59  | 6000  | 1.5167          | 0.7058   |
| 1.6592        | 0.64  | 6500  | 1.5083          | 0.7069   |
| 1.6386        | 0.69  | 7000  | 1.5050          | 0.7071   |
| 1.6449        | 0.74  | 7500  | 1.5033          | 0.7078   |
| 1.6362        | 0.79  | 8000  | 1.5032          | 0.7073   |
| 1.6337        | 0.84  | 8500  | 1.4978          | 0.7082   |
| 1.622         | 0.89  | 9000  | 1.4971          | 0.7089   |
| 1.6279        | 0.94  | 9500  | 1.4919          | 0.7094   |
| 1.6199        | 0.98  | 10000 | 1.4900          | 0.7096   |
| 1.6218        | 1.03  | 10500 | 1.4914          | 0.7099   |
| 1.6144        | 1.08  | 11000 | 1.4814          | 0.7113   |
| 1.6017        | 1.13  | 11500 | 1.4789          | 0.7113   |
| 1.6092        | 1.18  | 12000 | 1.4755          | 0.7119   |
| 1.6083        | 1.23  | 12500 | 1.4766          | 0.7119   |
| 1.6081        | 1.28  | 13000 | 1.4791          | 0.7113   |
| 1.615         | 1.33  | 13500 | 1.4722          | 0.7127   |
| 1.605         | 1.38  | 14000 | 1.4725          | 0.7124   |
| 1.598         | 1.43  | 14500 | 1.4763          | 0.7119   |
| 1.6004        | 1.48  | 15000 | 1.4661          | 0.7132   |
| 1.6074        | 1.53  | 15500 | 1.4713          | 0.7123   |
| 1.603         | 1.58  | 16000 | 1.4658          | 0.7135   |
| 1.5928        | 1.62  | 16500 | 1.4646          | 0.7135   |
| 1.5942        | 1.67  | 17000 | 1.4676          | 0.7132   |
| 1.5914        | 1.72  | 17500 | 1.4604          | 0.7145   |
| 1.5931        | 1.77  | 18000 | 1.4580          | 0.7142   |
| 1.5808        | 1.82  | 18500 | 1.4606          | 0.7148   |
| 1.5911        | 1.87  | 19000 | 1.4592          | 0.7143   |
| 1.5842        | 1.92  | 19500 | 1.4622          | 0.7143   |
| 1.5838        | 1.97  | 20000 | 1.4584          | 0.7146   |
| 1.585         | 2.02  | 20500 | 1.4630          | 0.7139   |
| 1.5772        | 2.07  | 21000 | 1.4557          | 0.7146   |
| 1.589         | 2.12  | 21500 | 1.4556          | 0.7151   |
| 1.5653        | 2.17  | 22000 | 1.4533          | 0.7154   |
| 1.5753        | 2.22  | 22500 | 1.4574          | 0.7152   |
| 1.5707        | 2.26  | 23000 | 1.4531          | 0.7155   |
| 1.5744        | 2.31  | 23500 | 1.4534          | 0.7153   |
| 1.5741        | 2.36  | 24000 | 1.4531          | 0.7155   |
| 1.5696        | 2.41  | 24500 | 1.4480          | 0.7161   |
| 1.5844        | 2.46  | 25000 | 1.4526          | 0.7157   |
| 1.576         | 2.51  | 25500 | 1.4478          | 0.7160   |
| 1.5621        | 2.56  | 26000 | 1.4497          | 0.7158   |
| 1.5707        | 2.61  | 26500 | 1.4514          | 0.7159   |
| 1.5819        | 2.66  | 27000 | 1.4478          | 0.7164   |
| 1.5663        | 2.71  | 27500 | 1.4503          | 0.7159   |
| 1.5834        | 2.76  | 28000 | 1.4507          | 0.7160   |
| 1.5726        | 2.81  | 28500 | 1.4426          | 0.7165   |
| 1.5695        | 2.86  | 29000 | 1.4466          | 0.7166   |
| 1.5791        | 2.9   | 29500 | 1.4464          | 0.7167   |
| 1.5711        | 2.95  | 30000 | 1.4434          | 0.7170   |
| 1.5724        | 3.0   | 30500 | 1.4423          | 0.7169   |
| 1.5648        | 3.05  | 31000 | 1.4410          | 0.7173   |
| 1.5666        | 3.1   | 31500 | 1.4427          | 0.7168   |
| 1.567         | 3.15  | 32000 | 1.4424          | 0.7170   |
| 1.5569        | 3.2   | 32500 | 1.4441          | 0.7168   |
| 1.5717        | 3.25  | 33000 | 1.4385          | 0.7182   |
| 1.5585        | 3.3   | 33500 | 1.4416          | 0.7175   |
| 1.5572        | 3.35  | 34000 | 1.4415          | 0.7174   |
| 1.5585        | 3.4   | 34500 | 1.4368          | 0.7181   |
| 1.5705        | 3.45  | 35000 | 1.4400          | 0.7178   |
| 1.5712        | 3.5   | 35500 | 1.4420          | 0.7173   |
| 1.5651        | 3.55  | 36000 | 1.4355          | 0.7186   |
| 1.5595        | 3.59  | 36500 | 1.4363          | 0.7179   |
| 1.5613        | 3.64  | 37000 | 1.4385          | 0.7179   |
| 1.5594        | 3.69  | 37500 | 1.4422          | 0.7172   |
| 1.5574        | 3.74  | 38000 | 1.4370          | 0.7179   |
| 1.5557        | 3.79  | 38500 | 1.4410          | 0.7177   |
| 1.5649        | 3.84  | 39000 | 1.4349          | 0.7181   |
| 1.5635        | 3.89  | 39500 | 1.4406          | 0.7174   |
| 1.5569        | 3.94  | 40000 | 1.4362          | 0.7182   |
| 1.5661        | 3.99  | 40500 | 1.4369          | 0.7180   |
| 1.5612        | 4.04  | 41000 | 1.4380          | 0.7178   |
| 1.5632        | 4.09  | 41500 | 1.4374          | 0.7180   |
| 1.5617        | 4.14  | 42000 | 1.4374          | 0.7178   |
| 1.5452        | 4.19  | 42500 | 1.4341          | 0.7185   |
| 1.5644        | 4.23  | 43000 | 1.4358          | 0.7181   |
| 1.5448        | 4.28  | 43500 | 1.4392          | 0.7178   |
| 1.559         | 4.33  | 44000 | 1.4363          | 0.7180   |
| 1.5599        | 4.38  | 44500 | 1.4332          | 0.7185   |
| 1.5586        | 4.43  | 45000 | 1.4391          | 0.7179   |
| 1.5527        | 4.48  | 45500 | 1.4355          | 0.7183   |
| 1.5592        | 4.53  | 46000 | 1.4314          | 0.7187   |
| 1.55          | 4.58  | 46500 | 1.4352          | 0.7179   |
| 1.5592        | 4.63  | 47000 | 1.4367          | 0.7180   |
| 1.5586        | 4.68  | 47500 | 1.4346          | 0.7183   |
| 1.5547        | 4.73  | 48000 | 1.4321          | 0.7192   |
| 1.5572        | 4.78  | 48500 | 1.4397          | 0.7174   |
| 1.5537        | 4.83  | 49000 | 1.4334          | 0.7186   |
| 1.5546        | 4.87  | 49500 | 1.4347          | 0.7181   |
| 1.5518        | 4.92  | 50000 | 1.4365          | 0.7183   |


### Framework versions

- Transformers 4.36.2
- Pytorch 2.0.0
- Datasets 2.15.0
- Tokenizers 0.15.0
