tensorboard generated_from_trainer base_model:bert-base-multilingual-cased license:apache-2.0 region:us
---
license: apache-2.0
base_model: bert-base-multilingual-cased
tags:
- generated_from_trainer
metrics:
- accuracy
model-index:
- name: ms
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# ms

This model is an adapter fine-tuned on top of [bert-base-multilingual-cased](https://huggingface.co/bert-base-multilingual-cased) on the Indonesian ConceptNet dataset.
It achieves the following results on the evaluation set:
- Loss: 1.1066
- Accuracy: 0.8008

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 5e-05
- train_batch_size: 16
- eval_batch_size: 16
- seed: 42
- distributed_type: multi-GPU
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- training_steps: 50000

### Training results

| Training Loss | Epoch | Step  | Validation Loss | Accuracy |
|:-------------:|:-----:|:-----:|:---------------:|:--------:|
| 2.7363        | 0.2   | 500   | 2.1330          | 0.6571   |
| 2.1199        | 0.4   | 1000  | 1.7464          | 0.7051   |
| 1.914         | 0.6   | 1500  | 1.7127          | 0.7092   |
| 1.8396        | 0.8   | 2000  | 1.5692          | 0.7292   |
| 1.7076        | 1.01  | 2500  | 1.5975          | 0.7296   |
| 1.6085        | 1.21  | 3000  | 1.4834          | 0.7376   |
| 1.5964        | 1.41  | 3500  | 1.4537          | 0.7450   |
| 1.5719        | 1.61  | 4000  | 1.4702          | 0.7417   |
| 1.594         | 1.81  | 4500  | 1.4020          | 0.7543   |
| 1.5542        | 2.01  | 5000  | 1.4244          | 0.7516   |
| 1.4969        | 2.21  | 5500  | 1.4354          | 0.7492   |
| 1.5142        | 2.41  | 6000  | 1.3493          | 0.7563   |
| 1.5101        | 2.61  | 6500  | 1.3877          | 0.7578   |
| 1.5271        | 2.82  | 7000  | 1.3608          | 0.7514   |
| 1.4514        | 3.02  | 7500  | 1.3714          | 0.7556   |
| 1.5128        | 3.22  | 8000  | 1.3200          | 0.7617   |
| 1.4385        | 3.42  | 8500  | 1.3858          | 0.7610   |
| 1.4548        | 3.62  | 9000  | 1.2575          | 0.7793   |
| 1.3884        | 3.82  | 9500  | 1.3005          | 0.7709   |
| 1.3708        | 4.02  | 10000 | 1.3500          | 0.7586   |
| 1.3538        | 4.22  | 10500 | 1.2987          | 0.7653   |
| 1.3822        | 4.42  | 11000 | 1.3071          | 0.7576   |
| 1.4111        | 4.63  | 11500 | 1.3216          | 0.7652   |
| 1.3212        | 4.83  | 12000 | 1.3120          | 0.7737   |
| 1.3805        | 5.03  | 12500 | 1.2815          | 0.7680   |
| 1.3255        | 5.23  | 13000 | 1.3030          | 0.7694   |
| 1.3636        | 5.43  | 13500 | 1.2303          | 0.7805   |
| 1.3774        | 5.63  | 14000 | 1.3069          | 0.7696   |
| 1.3382        | 5.83  | 14500 | 1.2284          | 0.7751   |
| 1.3301        | 6.03  | 15000 | 1.1942          | 0.7805   |
| 1.3062        | 6.23  | 15500 | 1.2433          | 0.7719   |
| 1.3294        | 6.44  | 16000 | 1.1711          | 0.7877   |
| 1.2573        | 6.64  | 16500 | 1.2233          | 0.7757   |
| 1.2834        | 6.84  | 17000 | 1.2187          | 0.7765   |
| 1.2821        | 7.04  | 17500 | 1.2685          | 0.7665   |
| 1.2397        | 7.24  | 18000 | 1.2535          | 0.7772   |
| 1.2888        | 7.44  | 18500 | 1.2106          | 0.7777   |
| 1.2544        | 7.64  | 19000 | 1.2491          | 0.7765   |
| 1.2741        | 7.84  | 19500 | 1.1817          | 0.7899   |
| 1.2456        | 8.05  | 20000 | 1.2064          | 0.7817   |
| 1.2842        | 8.25  | 20500 | 1.2037          | 0.7840   |
| 1.2406        | 8.45  | 21000 | 1.2503          | 0.7781   |
| 1.2128        | 8.65  | 21500 | 1.2035          | 0.7790   |
| 1.215         | 8.85  | 22000 | 1.2080          | 0.7813   |
| 1.2414        | 9.05  | 22500 | 1.1959          | 0.7837   |
| 1.247         | 9.25  | 23000 | 1.2165          | 0.7805   |
| 1.2174        | 9.45  | 23500 | 1.1872          | 0.7912   |
| 1.2556        | 9.65  | 24000 | 1.1676          | 0.7845   |
| 1.2282        | 9.86  | 24500 | 1.2111          | 0.7828   |
| 1.2167        | 10.06 | 25000 | 1.2313          | 0.7796   |
| 1.2444        | 10.26 | 25500 | 1.2200          | 0.7820   |
| 1.2461        | 10.46 | 26000 | 1.1966          | 0.7906   |
| 1.1735        | 10.66 | 26500 | 1.1616          | 0.7872   |
| 1.1832        | 10.86 | 27000 | 1.1214          | 0.7913   |
| 1.2018        | 11.06 | 27500 | 1.2279          | 0.7760   |
| 1.2119        | 11.26 | 28000 | 1.1877          | 0.7809   |
| 1.2142        | 11.46 | 28500 | 1.1578          | 0.7892   |
| 1.207         | 11.67 | 29000 | 1.1053          | 0.7978   |
| 1.1598        | 11.87 | 29500 | 1.1420          | 0.7948   |
| 1.2178        | 12.07 | 30000 | 1.1344          | 0.7938   |
| 1.2317        | 12.27 | 30500 | 1.1227          | 0.8026   |
| 1.1923        | 12.47 | 31000 | 1.1248          | 0.7942   |
| 1.1845        | 12.67 | 31500 | 1.1543          | 0.7923   |
| 1.2239        | 12.87 | 32000 | 1.1139          | 0.7936   |
| 1.1839        | 13.07 | 32500 | 1.1193          | 0.7938   |
| 1.2096        | 13.27 | 33000 | 1.1529          | 0.7946   |
| 1.1588        | 13.48 | 33500 | 1.1205          | 0.7952   |
| 1.151         | 13.68 | 34000 | 1.0998          | 0.7977   |
| 1.1638        | 13.88 | 34500 | 1.1626          | 0.7904   |
| 1.1729        | 14.08 | 35000 | 1.1781          | 0.7857   |
| 1.184         | 14.28 | 35500 | 1.1636          | 0.7959   |
| 1.1382        | 14.48 | 36000 | 1.1124          | 0.7916   |
| 1.1778        | 14.68 | 36500 | 1.1797          | 0.7888   |
| 1.1729        | 14.88 | 37000 | 1.1000          | 0.7978   |
| 1.1923        | 15.08 | 37500 | 1.1507          | 0.7926   |
| 1.1339        | 15.29 | 38000 | 1.1142          | 0.7992   |
| 1.1232        | 15.49 | 38500 | 1.1047          | 0.8021   |
| 1.1686        | 15.69 | 39000 | 1.0934          | 0.7938   |
| 1.123         | 15.89 | 39500 | 1.1268          | 0.7936   |
| 1.1099        | 16.09 | 40000 | 1.1262          | 0.7953   |
| 1.1196        | 16.29 | 40500 | 1.1442          | 0.7940   |
| 1.1594        | 16.49 | 41000 | 1.1250          | 0.7985   |
| 1.1272        | 16.69 | 41500 | 1.1059          | 0.7959   |
| 1.1219        | 16.89 | 42000 | 1.1019          | 0.7994   |
| 1.1423        | 17.1  | 42500 | 1.1140          | 0.7957   |
| 1.1648        | 17.3  | 43000 | 1.1530          | 0.7958   |
| 1.0872        | 17.5  | 43500 | 1.1235          | 0.7953   |
| 1.0844        | 17.7  | 44000 | 1.1448          | 0.7950   |
| 1.1169        | 17.9  | 44500 | 1.1408          | 0.7970   |
| 1.1109        | 18.1  | 45000 | 1.0774          | 0.8008   |
| 1.1427        | 18.3  | 45500 | 1.0881          | 0.8032   |
| 1.1231        | 18.5  | 46000 | 1.1551          | 0.7906   |
| 1.1591        | 18.7  | 46500 | 1.0803          | 0.8035   |
| 1.107         | 18.91 | 47000 | 1.0641          | 0.8053   |
| 1.1349        | 19.11 | 47500 | 1.1697          | 0.7886   |
| 1.1131        | 19.31 | 48000 | 1.1149          | 0.7932   |
| 1.1568        | 19.51 | 48500 | 1.1659          | 0.7889   |
| 1.1301        | 19.71 | 49000 | 1.0934          | 0.8005   |
| 1.1609        | 19.91 | 49500 | 1.0778          | 0.8059   |
| 1.0941        | 20.11 | 50000 | 1.1083          | 0.7971   |


### Framework versions

- Transformers 4.35.2
- Pytorch 2.0.0
- Datasets 2.15.0
- Tokenizers 0.15.0
