tensorboard generated_from_trainer base_model:bert-base-multilingual-cased license:apache-2.0 region:us
---
license: apache-2.0
base_model: bert-base-multilingual-cased
tags:
- generated_from_trainer
metrics:
- accuracy
model-index:
- name: mt
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# mt

This model is an adapter fine-tuned on top of of [bert-base-multilingual-cased](https://huggingface.co/bert-base-multilingual-cased) on the Maltese ConceptNet dataset.
It achieves the following results on the evaluation set:
- Loss: 0.8117
- Accuracy: 0.8590

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 5e-05
- train_batch_size: 16
- eval_batch_size: 16
- seed: 42
- distributed_type: multi-GPU
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- training_steps: 50000

### Training results

| Training Loss | Epoch  | Step  | Validation Loss | Accuracy |
|:-------------:|:------:|:-----:|:---------------:|:--------:|
| 2.6689        | 1.04   | 500   | 2.1345          | 0.6677   |
| 2.1415        | 2.07   | 1000  | 1.8436          | 0.6926   |
| 1.9421        | 3.11   | 1500  | 1.7874          | 0.6907   |
| 1.7588        | 4.14   | 2000  | 1.7605          | 0.7013   |
| 1.6729        | 5.18   | 2500  | 1.7568          | 0.6957   |
| 1.596         | 6.21   | 3000  | 1.5006          | 0.7273   |
| 1.5778        | 7.25   | 3500  | 1.3924          | 0.7451   |
| 1.4821        | 8.28   | 4000  | 1.6097          | 0.7099   |
| 1.4183        | 9.32   | 4500  | 1.3552          | 0.7491   |
| 1.4197        | 10.35  | 5000  | 1.2847          | 0.7513   |
| 1.3156        | 11.39  | 5500  | 1.3173          | 0.7496   |
| 1.2882        | 12.42  | 6000  | 1.2817          | 0.7738   |
| 1.2692        | 13.46  | 6500  | 1.1892          | 0.7751   |
| 1.2368        | 14.49  | 7000  | 1.2363          | 0.7816   |
| 1.1975        | 15.53  | 7500  | 1.2442          | 0.7700   |
| 1.1907        | 16.56  | 8000  | 1.2569          | 0.7720   |
| 1.1231        | 17.6   | 8500  | 1.1386          | 0.7761   |
| 1.0873        | 18.63  | 9000  | 1.2105          | 0.7856   |
| 1.1242        | 19.67  | 9500  | 1.2142          | 0.7738   |
| 1.0367        | 20.7   | 10000 | 1.2121          | 0.7712   |
| 1.0869        | 21.74  | 10500 | 1.0782          | 0.7955   |
| 1.0353        | 22.77  | 11000 | 0.9918          | 0.8069   |
| 1.0324        | 23.81  | 11500 | 1.0908          | 0.7971   |
| 1.0145        | 24.84  | 12000 | 1.0945          | 0.7975   |
| 0.9951        | 25.88  | 12500 | 1.0005          | 0.8028   |
| 0.9483        | 26.92  | 13000 | 0.9638          | 0.8187   |
| 0.9304        | 27.95  | 13500 | 0.9761          | 0.8205   |
| 0.8835        | 28.99  | 14000 | 1.0620          | 0.8046   |
| 0.9097        | 30.02  | 14500 | 0.9138          | 0.8060   |
| 0.9293        | 31.06  | 15000 | 0.9180          | 0.8176   |
| 0.9043        | 32.09  | 15500 | 0.9215          | 0.8208   |
| 0.8581        | 33.13  | 16000 | 0.9625          | 0.8225   |
| 0.8638        | 34.16  | 16500 | 0.8586          | 0.8368   |
| 0.874         | 35.2   | 17000 | 1.0044          | 0.8135   |
| 0.8235        | 36.23  | 17500 | 0.9755          | 0.8184   |
| 0.8589        | 37.27  | 18000 | 0.9042          | 0.8292   |
| 0.8107        | 38.3   | 18500 | 0.8821          | 0.8272   |
| 0.8346        | 39.34  | 19000 | 0.9061          | 0.8248   |
| 0.8393        | 40.37  | 19500 | 0.9796          | 0.8235   |
| 0.789         | 41.41  | 20000 | 0.9015          | 0.8331   |
| 0.8121        | 42.44  | 20500 | 0.8589          | 0.8386   |
| 0.7709        | 43.48  | 21000 | 0.8836          | 0.8351   |
| 0.7922        | 44.51  | 21500 | 0.9524          | 0.8180   |
| 0.7457        | 45.55  | 22000 | 0.8350          | 0.8364   |
| 0.7386        | 46.58  | 22500 | 0.9025          | 0.8341   |
| 0.7515        | 47.62  | 23000 | 0.9092          | 0.8390   |
| 0.7324        | 48.65  | 23500 | 0.8322          | 0.8421   |
| 0.7314        | 49.69  | 24000 | 0.7968          | 0.8477   |
| 0.7442        | 50.72  | 24500 | 0.9305          | 0.8324   |
| 0.7074        | 51.76  | 25000 | 1.0011          | 0.8208   |
| 0.739         | 52.8   | 25500 | 0.8732          | 0.8331   |
| 0.7243        | 53.83  | 26000 | 0.7857          | 0.8480   |
| 0.6842        | 54.87  | 26500 | 0.7945          | 0.8377   |
| 0.6991        | 55.9   | 27000 | 0.9628          | 0.8275   |
| 0.6896        | 56.94  | 27500 | 0.8363          | 0.8410   |
| 0.6925        | 57.97  | 28000 | 0.8433          | 0.8392   |
| 0.7081        | 59.01  | 28500 | 1.0086          | 0.8223   |
| 0.6598        | 60.04  | 29000 | 0.9251          | 0.8333   |
| 0.6677        | 61.08  | 29500 | 0.8823          | 0.8437   |
| 0.695         | 62.11  | 30000 | 0.7751          | 0.8560   |
| 0.7108        | 63.15  | 30500 | 0.8452          | 0.8481   |
| 0.6721        | 64.18  | 31000 | 0.8560          | 0.8413   |
| 0.6571        | 65.22  | 31500 | 0.9800          | 0.8163   |
| 0.6891        | 66.25  | 32000 | 0.8106          | 0.8457   |
| 0.6541        | 67.29  | 32500 | 0.8197          | 0.8430   |
| 0.6559        | 68.32  | 33000 | 0.8678          | 0.8388   |
| 0.6554        | 69.36  | 33500 | 0.7396          | 0.8662   |
| 0.618         | 70.39  | 34000 | 0.8518          | 0.8376   |
| 0.6558        | 71.43  | 34500 | 0.7706          | 0.8409   |
| 0.6034        | 72.46  | 35000 | 0.7829          | 0.8518   |
| 0.6336        | 73.5   | 35500 | 0.7835          | 0.8591   |
| 0.6287        | 74.53  | 36000 | 0.7548          | 0.8575   |
| 0.6065        | 75.57  | 36500 | 0.8542          | 0.8508   |
| 0.6029        | 76.6   | 37000 | 0.8203          | 0.8405   |
| 0.6208        | 77.64  | 37500 | 0.7082          | 0.8661   |
| 0.64          | 78.67  | 38000 | 0.8505          | 0.8410   |
| 0.6144        | 79.71  | 38500 | 0.7246          | 0.8604   |
| 0.6507        | 80.75  | 39000 | 0.7150          | 0.8611   |
| 0.6177        | 81.78  | 39500 | 0.9332          | 0.84     |
| 0.6159        | 82.82  | 40000 | 0.6427          | 0.8733   |
| 0.5944        | 83.85  | 40500 | 0.7721          | 0.8411   |
| 0.6044        | 84.89  | 41000 | 0.8968          | 0.8449   |
| 0.6           | 85.92  | 41500 | 0.7673          | 0.8538   |
| 0.5899        | 86.96  | 42000 | 0.8039          | 0.8505   |
| 0.5812        | 87.99  | 42500 | 0.7467          | 0.8567   |
| 0.5977        | 89.03  | 43000 | 0.9534          | 0.8316   |
| 0.6019        | 90.06  | 43500 | 0.9170          | 0.8316   |
| 0.563         | 91.1   | 44000 | 0.7761          | 0.8569   |
| 0.6347        | 92.13  | 44500 | 0.7811          | 0.8577   |
| 0.5855        | 93.17  | 45000 | 0.7562          | 0.8606   |
| 0.6026        | 94.2   | 45500 | 0.7490          | 0.8636   |
| 0.5846        | 95.24  | 46000 | 0.7456          | 0.8487   |
| 0.5635        | 96.27  | 46500 | 0.8115          | 0.8495   |
| 0.5903        | 97.31  | 47000 | 0.8137          | 0.8448   |
| 0.576         | 98.34  | 47500 | 0.8441          | 0.8424   |
| 0.5745        | 99.38  | 48000 | 0.7266          | 0.8609   |
| 0.5915        | 100.41 | 48500 | 0.9169          | 0.8446   |
| 0.601         | 101.45 | 49000 | 0.7671          | 0.8576   |
| 0.5713        | 102.48 | 49500 | 0.7868          | 0.8487   |
| 0.5541        | 103.52 | 50000 | 0.7907          | 0.8569   |


### Framework versions

- Transformers 4.35.2
- Pytorch 2.0.0
- Datasets 2.15.0
- Tokenizers 0.15.0
