tensorboard generated_from_trainer base_model:bert-base-multilingual-cased license:apache-2.0 region:us
---
license: apache-2.0
base_model: bert-base-multilingual-cased
tags:
- generated_from_trainer
metrics:
- accuracy
model-index:
- name: bg
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# bg

This model is an adapter fine-tuned on top of [bert-base-multilingual-cased](https://huggingface.co/bert-base-multilingual-cased) on the Bulgarian ConceptNet dataset.
It achieves the following results on the evaluation set:
- Loss: 0.4640
- Accuracy: 0.8875

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 5e-05
- train_batch_size: 16
- eval_batch_size: 16
- seed: 42
- distributed_type: multi-GPU
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- training_steps: 50000

### Training results

| Training Loss | Epoch | Step  | Validation Loss | Accuracy |
|:-------------:|:-----:|:-----:|:---------------:|:--------:|
| 1.5057        | 0.15  | 500   | 0.9846          | 0.8149   |
| 1.0172        | 0.31  | 1000  | 0.8395          | 0.8259   |
| 0.8814        | 0.46  | 1500  | 0.7823          | 0.8368   |
| 0.8405        | 0.61  | 2000  | 0.7437          | 0.8449   |
| 0.7773        | 0.77  | 2500  | 0.7247          | 0.8387   |
| 0.7762        | 0.92  | 3000  | 0.6521          | 0.8513   |
| 0.7186        | 1.07  | 3500  | 0.6834          | 0.8492   |
| 0.7033        | 1.22  | 4000  | 0.6715          | 0.8523   |
| 0.672         | 1.38  | 4500  | 0.6539          | 0.8560   |
| 0.6613        | 1.53  | 5000  | 0.6387          | 0.8567   |
| 0.6712        | 1.68  | 5500  | 0.6180          | 0.8624   |
| 0.6776        | 1.84  | 6000  | 0.6635          | 0.8537   |
| 0.6484        | 1.99  | 6500  | 0.5946          | 0.8661   |
| 0.6817        | 2.14  | 7000  | 0.6126          | 0.8655   |
| 0.6392        | 2.3   | 7500  | 0.6136          | 0.8613   |
| 0.6394        | 2.45  | 8000  | 0.6321          | 0.8621   |
| 0.6273        | 2.6   | 8500  | 0.5997          | 0.8629   |
| 0.5993        | 2.76  | 9000  | 0.6028          | 0.8646   |
| 0.6527        | 2.91  | 9500  | 0.6584          | 0.8510   |
| 0.5897        | 3.06  | 10000 | 0.5728          | 0.8676   |
| 0.574         | 3.21  | 10500 | 0.5870          | 0.8671   |
| 0.6026        | 3.37  | 11000 | 0.6067          | 0.8677   |
| 0.5896        | 3.52  | 11500 | 0.6000          | 0.8638   |
| 0.566         | 3.67  | 12000 | 0.5566          | 0.8712   |
| 0.5928        | 3.83  | 12500 | 0.5621          | 0.8675   |
| 0.597         | 3.98  | 13000 | 0.5162          | 0.8771   |
| 0.5836        | 4.13  | 13500 | 0.5498          | 0.8696   |
| 0.5864        | 4.29  | 14000 | 0.5728          | 0.8640   |
| 0.5562        | 4.44  | 14500 | 0.6000          | 0.8623   |
| 0.5999        | 4.59  | 15000 | 0.5589          | 0.8679   |
| 0.5767        | 4.75  | 15500 | 0.5713          | 0.8681   |
| 0.5574        | 4.9   | 16000 | 0.5338          | 0.8739   |
| 0.568         | 5.05  | 16500 | 0.5527          | 0.8725   |
| 0.5568        | 5.21  | 17000 | 0.5058          | 0.8777   |
| 0.5369        | 5.36  | 17500 | 0.5599          | 0.8720   |
| 0.518         | 5.51  | 18000 | 0.5610          | 0.8720   |
| 0.5637        | 5.66  | 18500 | 0.5467          | 0.8728   |
| 0.557         | 5.82  | 19000 | 0.5349          | 0.8714   |
| 0.5499        | 5.97  | 19500 | 0.5468          | 0.8724   |
| 0.5304        | 6.12  | 20000 | 0.5243          | 0.8741   |
| 0.5431        | 6.28  | 20500 | 0.4998          | 0.8784   |
| 0.5508        | 6.43  | 21000 | 0.5367          | 0.8764   |
| 0.5701        | 6.58  | 21500 | 0.5365          | 0.8734   |
| 0.521         | 6.74  | 22000 | 0.4879          | 0.8819   |
| 0.5514        | 6.89  | 22500 | 0.5106          | 0.8787   |
| 0.547         | 7.04  | 23000 | 0.5258          | 0.8747   |
| 0.5512        | 7.2   | 23500 | 0.4975          | 0.8778   |
| 0.5407        | 7.35  | 24000 | 0.4944          | 0.8786   |
| 0.5181        | 7.5   | 24500 | 0.4912          | 0.8795   |
| 0.5493        | 7.65  | 25000 | 0.5188          | 0.8730   |
| 0.5388        | 7.81  | 25500 | 0.5000          | 0.8831   |
| 0.5284        | 7.96  | 26000 | 0.5161          | 0.8737   |
| 0.5116        | 8.11  | 26500 | 0.5263          | 0.8760   |
| 0.5161        | 8.27  | 27000 | 0.5002          | 0.8787   |
| 0.5185        | 8.42  | 27500 | 0.5127          | 0.8745   |
| 0.5291        | 8.57  | 28000 | 0.5116          | 0.8782   |
| 0.5061        | 8.73  | 28500 | 0.4972          | 0.8774   |
| 0.479         | 8.88  | 29000 | 0.4978          | 0.8798   |
| 0.5154        | 9.03  | 29500 | 0.5088          | 0.8771   |
| 0.4989        | 9.19  | 30000 | 0.5119          | 0.8744   |
| 0.5098        | 9.34  | 30500 | 0.4916          | 0.8826   |
| 0.4777        | 9.49  | 31000 | 0.4957          | 0.8824   |
| 0.5462        | 9.64  | 31500 | 0.4846          | 0.8779   |
| 0.509         | 9.8   | 32000 | 0.4873          | 0.8810   |
| 0.5181        | 9.95  | 32500 | 0.5227          | 0.8710   |
| 0.5269        | 10.1  | 33000 | 0.4929          | 0.8803   |
| 0.5094        | 10.26 | 33500 | 0.4841          | 0.8877   |
| 0.5033        | 10.41 | 34000 | 0.5129          | 0.8805   |
| 0.4913        | 10.56 | 34500 | 0.4978          | 0.8789   |
| 0.4938        | 10.72 | 35000 | 0.4640          | 0.8838   |
| 0.4954        | 10.87 | 35500 | 0.4991          | 0.8794   |
| 0.458         | 11.02 | 36000 | 0.4453          | 0.8886   |
| 0.526         | 11.18 | 36500 | 0.4863          | 0.8832   |
| 0.4809        | 11.33 | 37000 | 0.4923          | 0.8784   |
| 0.466         | 11.48 | 37500 | 0.4824          | 0.8807   |
| 0.4903        | 11.64 | 38000 | 0.4552          | 0.8848   |
| 0.4875        | 11.79 | 38500 | 0.4850          | 0.8780   |
| 0.4858        | 11.94 | 39000 | 0.4728          | 0.8833   |
| 0.4868        | 12.09 | 39500 | 0.4868          | 0.8800   |
| 0.485         | 12.25 | 40000 | 0.4935          | 0.8802   |
| 0.4823        | 12.4  | 40500 | 0.4789          | 0.8828   |
| 0.4629        | 12.55 | 41000 | 0.4834          | 0.8835   |
| 0.4915        | 12.71 | 41500 | 0.4864          | 0.8812   |
| 0.473         | 12.86 | 42000 | 0.5136          | 0.8793   |
| 0.4849        | 13.01 | 42500 | 0.4823          | 0.8815   |
| 0.4582        | 13.17 | 43000 | 0.4637          | 0.8844   |
| 0.4938        | 13.32 | 43500 | 0.4829          | 0.8842   |
| 0.4682        | 13.47 | 44000 | 0.4799          | 0.8817   |
| 0.4885        | 13.63 | 44500 | 0.4754          | 0.8858   |
| 0.4641        | 13.78 | 45000 | 0.4738          | 0.8849   |
| 0.4664        | 13.93 | 45500 | 0.4512          | 0.8869   |
| 0.4722        | 14.08 | 46000 | 0.4821          | 0.8836   |
| 0.485         | 14.24 | 46500 | 0.4735          | 0.8842   |
| 0.4784        | 14.39 | 47000 | 0.4557          | 0.8823   |
| 0.4821        | 14.54 | 47500 | 0.4707          | 0.8856   |
| 0.478         | 14.7  | 48000 | 0.4682          | 0.8846   |
| 0.451         | 14.85 | 48500 | 0.4744          | 0.8781   |
| 0.4582        | 15.0  | 49000 | 0.4617          | 0.8835   |
| 0.4949        | 15.16 | 49500 | 0.4769          | 0.8835   |
| 0.4546        | 15.31 | 50000 | 0.4677          | 0.8835   |


### Framework versions

- Transformers 4.35.2
- Pytorch 2.0.0
- Datasets 2.15.0
- Tokenizers 0.15.0
