transformers pytorch xlm-roberta generated_from_trainer dataset:massive base_model:FacebookAI/xlm-roberta-base license:mit endpoints_compatible region:us
---
license: mit
base_model: FacebookAI/xlm-roberta-base
tags:
- generated_from_trainer
datasets:
- massive
metrics:
- accuracy
- f1
model-index:
- name: scenario-KD-SCR-MSV-EN-EN-D2_data-en-massive_all_1_1
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# scenario-KD-SCR-MSV-EN-EN-D2_data-en-massive_all_1_1

This model is a fine-tuned version of [FacebookAI/xlm-roberta-base](https://huggingface.co/FacebookAI/xlm-roberta-base) on the massive dataset.
It achieves the following results on the evaluation set:
- Loss: 14.1819
- Accuracy: 0.1011
- F1: 0.0771

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 5e-05
- train_batch_size: 32
- eval_batch_size: 32
- seed: 47
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 30

### Training results

| Training Loss | Epoch | Step  | Validation Loss | Accuracy | F1     |
|:-------------:|:-----:|:-----:|:---------------:|:--------:|:------:|
| No log        | 0.28  | 100   | 9.3069          | 0.0775   | 0.0065 |
| No log        | 0.56  | 200   | 9.2256          | 0.0881   | 0.0285 |
| No log        | 0.83  | 300   | 9.5054          | 0.0874   | 0.0449 |
| No log        | 1.11  | 400   | 9.6152          | 0.0930   | 0.0575 |
| 6.7715        | 1.39  | 500   | 9.8484          | 0.1023   | 0.0669 |
| 6.7715        | 1.67  | 600   | 9.7053          | 0.1054   | 0.0676 |
| 6.7715        | 1.94  | 700   | 10.0033         | 0.1116   | 0.0761 |
| 6.7715        | 2.22  | 800   | 10.3355         | 0.1024   | 0.0758 |
| 6.7715        | 2.5   | 900   | 10.5199         | 0.1051   | 0.0756 |
| 3.5854        | 2.78  | 1000  | 10.4890         | 0.1003   | 0.0730 |
| 3.5854        | 3.06  | 1100  | 11.1371         | 0.1026   | 0.0742 |
| 3.5854        | 3.33  | 1200  | 10.6889         | 0.1114   | 0.0744 |
| 3.5854        | 3.61  | 1300  | 11.2190         | 0.1009   | 0.0723 |
| 3.5854        | 3.89  | 1400  | 11.6248         | 0.1006   | 0.0752 |
| 2.5619        | 4.17  | 1500  | 11.4919         | 0.1041   | 0.0773 |
| 2.5619        | 4.44  | 1600  | 11.9283         | 0.0969   | 0.0709 |
| 2.5619        | 4.72  | 1700  | 11.9070         | 0.0994   | 0.0729 |
| 2.5619        | 5.0   | 1800  | 11.9216         | 0.0989   | 0.0679 |
| 2.5619        | 5.28  | 1900  | 12.1528         | 0.1011   | 0.0714 |
| 1.9218        | 5.56  | 2000  | 12.8701         | 0.0963   | 0.0705 |
| 1.9218        | 5.83  | 2100  | 12.5439         | 0.0992   | 0.0741 |
| 1.9218        | 6.11  | 2200  | 14.2823         | 0.0970   | 0.0706 |
| 1.9218        | 6.39  | 2300  | 12.9570         | 0.0996   | 0.0701 |
| 1.9218        | 6.67  | 2400  | 12.7219         | 0.1098   | 0.0758 |
| 1.5444        | 6.94  | 2500  | 13.2803         | 0.0926   | 0.0704 |
| 1.5444        | 7.22  | 2600  | 13.5962         | 0.0999   | 0.0764 |
| 1.5444        | 7.5   | 2700  | 13.6673         | 0.0997   | 0.0743 |
| 1.5444        | 7.78  | 2800  | 13.8965         | 0.1047   | 0.0724 |
| 1.5444        | 8.06  | 2900  | 13.5849         | 0.0922   | 0.0728 |
| 1.2579        | 8.33  | 3000  | 15.0406         | 0.1019   | 0.0736 |
| 1.2579        | 8.61  | 3100  | 15.6149         | 0.1019   | 0.0738 |
| 1.2579        | 8.89  | 3200  | 14.3982         | 0.0980   | 0.0762 |
| 1.2579        | 9.17  | 3300  | 13.6922         | 0.1017   | 0.0651 |
| 1.2579        | 9.44  | 3400  | 13.9173         | 0.1048   | 0.0700 |
| 1.0753        | 9.72  | 3500  | 14.3425         | 0.1003   | 0.0742 |
| 1.0753        | 10.0  | 3600  | 14.0474         | 0.0955   | 0.0771 |
| 1.0753        | 10.28 | 3700  | 13.7067         | 0.1012   | 0.0756 |
| 1.0753        | 10.56 | 3800  | 13.5322         | 0.1067   | 0.0738 |
| 1.0753        | 10.83 | 3900  | 14.6059         | 0.0988   | 0.0763 |
| 0.9469        | 11.11 | 4000  | 15.2368         | 0.1030   | 0.0811 |
| 0.9469        | 11.39 | 4100  | 14.3134         | 0.0936   | 0.0713 |
| 0.9469        | 11.67 | 4200  | 14.9626         | 0.1018   | 0.0761 |
| 0.9469        | 11.94 | 4300  | 13.3460         | 0.0969   | 0.0697 |
| 0.9469        | 12.22 | 4400  | 13.9374         | 0.0912   | 0.0743 |
| 0.8089        | 12.5  | 4500  | 14.3300         | 0.1032   | 0.0765 |
| 0.8089        | 12.78 | 4600  | 15.1234         | 0.1022   | 0.0778 |
| 0.8089        | 13.06 | 4700  | 14.9212         | 0.0978   | 0.0763 |
| 0.8089        | 13.33 | 4800  | 13.7898         | 0.1019   | 0.0756 |
| 0.8089        | 13.61 | 4900  | 13.8005         | 0.0923   | 0.0752 |
| 0.7474        | 13.89 | 5000  | 13.4644         | 0.0887   | 0.0702 |
| 0.7474        | 14.17 | 5100  | 14.1735         | 0.0990   | 0.0762 |
| 0.7474        | 14.44 | 5200  | 14.0039         | 0.0948   | 0.0732 |
| 0.7474        | 14.72 | 5300  | 15.0009         | 0.0980   | 0.0758 |
| 0.7474        | 15.0  | 5400  | 15.7028         | 0.1025   | 0.0759 |
| 0.6826        | 15.28 | 5500  | 14.9519         | 0.0999   | 0.0759 |
| 0.6826        | 15.56 | 5600  | 14.4389         | 0.1038   | 0.0759 |
| 0.6826        | 15.83 | 5700  | 15.6013         | 0.0999   | 0.0787 |
| 0.6826        | 16.11 | 5800  | 14.1587         | 0.0992   | 0.0766 |
| 0.6826        | 16.39 | 5900  | 16.4603         | 0.0989   | 0.0773 |
| 0.6265        | 16.67 | 6000  | 16.0249         | 0.1035   | 0.0763 |
| 0.6265        | 16.94 | 6100  | 14.4053         | 0.1007   | 0.0780 |
| 0.6265        | 17.22 | 6200  | 13.9288         | 0.0946   | 0.0756 |
| 0.6265        | 17.5  | 6300  | 15.3227         | 0.1016   | 0.0768 |
| 0.6265        | 17.78 | 6400  | 13.8783         | 0.0948   | 0.0729 |
| 0.5821        | 18.06 | 6500  | 14.1279         | 0.0928   | 0.0745 |
| 0.5821        | 18.33 | 6600  | 14.4890         | 0.0961   | 0.0744 |
| 0.5821        | 18.61 | 6700  | 14.8176         | 0.1014   | 0.0760 |
| 0.5821        | 18.89 | 6800  | 14.7877         | 0.1012   | 0.0775 |
| 0.5821        | 19.17 | 6900  | 14.5485         | 0.1013   | 0.0786 |
| 0.5462        | 19.44 | 7000  | 15.0580         | 0.1007   | 0.0818 |
| 0.5462        | 19.72 | 7100  | 13.9213         | 0.0935   | 0.0738 |
| 0.5462        | 20.0  | 7200  | 16.5866         | 0.1005   | 0.0766 |
| 0.5462        | 20.28 | 7300  | 14.1027         | 0.0969   | 0.0748 |
| 0.5462        | 20.56 | 7400  | 14.2922         | 0.1021   | 0.0759 |
| 0.519         | 20.83 | 7500  | 14.2771         | 0.0986   | 0.0772 |
| 0.519         | 21.11 | 7600  | 14.2693         | 0.1027   | 0.0749 |
| 0.519         | 21.39 | 7700  | 13.5863         | 0.1042   | 0.0795 |
| 0.519         | 21.67 | 7800  | 15.6105         | 0.1018   | 0.0820 |
| 0.519         | 21.94 | 7900  | 15.7645         | 0.1024   | 0.0829 |
| 0.499         | 22.22 | 8000  | 14.1501         | 0.0988   | 0.0782 |
| 0.499         | 22.5  | 8100  | 13.9119         | 0.0988   | 0.0771 |
| 0.499         | 22.78 | 8200  | 14.5926         | 0.1031   | 0.0790 |
| 0.499         | 23.06 | 8300  | 14.6785         | 0.1011   | 0.0748 |
| 0.499         | 23.33 | 8400  | 13.7131         | 0.0977   | 0.0772 |
| 0.4814        | 23.61 | 8500  | 13.9351         | 0.1013   | 0.0773 |
| 0.4814        | 23.89 | 8600  | 13.9906         | 0.0979   | 0.0758 |
| 0.4814        | 24.17 | 8700  | 14.1299         | 0.1005   | 0.0758 |
| 0.4814        | 24.44 | 8800  | 13.1678         | 0.1007   | 0.0761 |
| 0.4814        | 24.72 | 8900  | 13.5850         | 0.0967   | 0.0753 |
| 0.4569        | 25.0  | 9000  | 14.9372         | 0.1032   | 0.0787 |
| 0.4569        | 25.28 | 9100  | 14.4079         | 0.1006   | 0.0767 |
| 0.4569        | 25.56 | 9200  | 14.3035         | 0.1016   | 0.0770 |
| 0.4569        | 25.83 | 9300  | 14.0566         | 0.1023   | 0.0773 |
| 0.4569        | 26.11 | 9400  | 14.0034         | 0.0994   | 0.0794 |
| 0.4514        | 26.39 | 9500  | 14.6733         | 0.0985   | 0.0792 |
| 0.4514        | 26.67 | 9600  | 14.3151         | 0.1010   | 0.0772 |
| 0.4514        | 26.94 | 9700  | 14.6124         | 0.1005   | 0.0796 |
| 0.4514        | 27.22 | 9800  | 13.5692         | 0.0985   | 0.0751 |
| 0.4514        | 27.5  | 9900  | 14.7810         | 0.0999   | 0.0790 |
| 0.4355        | 27.78 | 10000 | 14.6610         | 0.0981   | 0.0768 |
| 0.4355        | 28.06 | 10100 | 14.5152         | 0.0999   | 0.0800 |
| 0.4355        | 28.33 | 10200 | 14.1330         | 0.1001   | 0.0779 |
| 0.4355        | 28.61 | 10300 | 14.5362         | 0.0998   | 0.0777 |
| 0.4355        | 28.89 | 10400 | 14.1860         | 0.1004   | 0.0783 |
| 0.4302        | 29.17 | 10500 | 14.6424         | 0.1015   | 0.0789 |
| 0.4302        | 29.44 | 10600 | 14.3844         | 0.1010   | 0.0779 |
| 0.4302        | 29.72 | 10700 | 14.2358         | 0.1006   | 0.0781 |
| 0.4302        | 30.0  | 10800 | 14.1819         | 0.1011   | 0.0771 |


### Framework versions

- Transformers 4.33.3
- Pytorch 2.1.1+cu121
- Datasets 2.14.5
- Tokenizers 0.13.3
