transformers pytorch xlm-roberta generated_from_trainer dataset:massive base_model:FacebookAI/xlm-roberta-base license:mit endpoints_compatible region:us
---
license: mit
base_model: FacebookAI/xlm-roberta-base
tags:
- generated_from_trainer
datasets:
- massive
metrics:
- accuracy
- f1
model-index:
- name: scenario-KD-SCR-MSV-EN-CL-D2_data-en-massive_all_1_1
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# scenario-KD-SCR-MSV-EN-CL-D2_data-en-massive_all_1_1

This model is a fine-tuned version of [FacebookAI/xlm-roberta-base](https://huggingface.co/FacebookAI/xlm-roberta-base) on the massive dataset.
It achieves the following results on the evaluation set:
- Loss: 12.8207
- Accuracy: 0.1087
- F1: 0.0816

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 5e-05
- train_batch_size: 32
- eval_batch_size: 32
- seed: 47
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 30

### Training results

| Training Loss | Epoch | Step  | Validation Loss | Accuracy | F1     |
|:-------------:|:-----:|:-----:|:---------------:|:--------:|:------:|
| No log        | 0.28  | 100   | 9.4505          | 0.0729   | 0.0048 |
| No log        | 0.56  | 200   | 9.2984          | 0.0890   | 0.0261 |
| No log        | 0.83  | 300   | 9.7553          | 0.0848   | 0.0444 |
| No log        | 1.11  | 400   | 9.7570          | 0.1069   | 0.0697 |
| 6.4752        | 1.39  | 500   | 10.0436         | 0.1015   | 0.0695 |
| 6.4752        | 1.67  | 600   | 10.1380         | 0.1028   | 0.0676 |
| 6.4752        | 1.94  | 700   | 10.3695         | 0.1066   | 0.0728 |
| 6.4752        | 2.22  | 800   | 10.8331         | 0.0965   | 0.0689 |
| 6.4752        | 2.5   | 900   | 11.1266         | 0.0997   | 0.0749 |
| 3.2372        | 2.78  | 1000  | 10.8630         | 0.0989   | 0.0681 |
| 3.2372        | 3.06  | 1100  | 11.1994         | 0.1067   | 0.0769 |
| 3.2372        | 3.33  | 1200  | 10.9878         | 0.1034   | 0.0724 |
| 3.2372        | 3.61  | 1300  | 11.5239         | 0.1017   | 0.0719 |
| 3.2372        | 3.89  | 1400  | 11.1544         | 0.1120   | 0.0788 |
| 2.2795        | 4.17  | 1500  | 11.3500         | 0.0981   | 0.0717 |
| 2.2795        | 4.44  | 1600  | 11.5489         | 0.1016   | 0.0718 |
| 2.2795        | 4.72  | 1700  | 11.5448         | 0.1075   | 0.0810 |
| 2.2795        | 5.0   | 1800  | 12.4145         | 0.1075   | 0.0787 |
| 2.2795        | 5.28  | 1900  | 11.9629         | 0.1000   | 0.0712 |
| 1.7085        | 5.56  | 2000  | 12.7747         | 0.1037   | 0.0748 |
| 1.7085        | 5.83  | 2100  | 12.2893         | 0.1025   | 0.0742 |
| 1.7085        | 6.11  | 2200  | 13.1726         | 0.1026   | 0.0809 |
| 1.7085        | 6.39  | 2300  | 12.7693         | 0.0951   | 0.0697 |
| 1.7085        | 6.67  | 2400  | 12.2965         | 0.0988   | 0.0700 |
| 1.3908        | 6.94  | 2500  | 12.5426         | 0.0990   | 0.0683 |
| 1.3908        | 7.22  | 2600  | 13.1850         | 0.1011   | 0.0792 |
| 1.3908        | 7.5   | 2700  | 12.6008         | 0.1050   | 0.0745 |
| 1.3908        | 7.78  | 2800  | 13.5513         | 0.1061   | 0.0730 |
| 1.3908        | 8.06  | 2900  | 13.3849         | 0.1029   | 0.0755 |
| 1.1599        | 8.33  | 3000  | 13.0032         | 0.1039   | 0.0747 |
| 1.1599        | 8.61  | 3100  | 13.6328         | 0.1044   | 0.0769 |
| 1.1599        | 8.89  | 3200  | 13.3697         | 0.0955   | 0.0715 |
| 1.1599        | 9.17  | 3300  | 13.3708         | 0.1066   | 0.0766 |
| 1.1599        | 9.44  | 3400  | 13.1751         | 0.1062   | 0.0807 |
| 1.0188        | 9.72  | 3500  | 12.4242         | 0.1046   | 0.0724 |
| 1.0188        | 10.0  | 3600  | 13.0852         | 0.0974   | 0.0730 |
| 1.0188        | 10.28 | 3700  | 12.9191         | 0.1052   | 0.0757 |
| 1.0188        | 10.56 | 3800  | 12.6856         | 0.1036   | 0.0731 |
| 1.0188        | 10.83 | 3900  | 13.3897         | 0.1035   | 0.0762 |
| 0.9195        | 11.11 | 4000  | 13.7717         | 0.1059   | 0.0768 |
| 0.9195        | 11.39 | 4100  | 12.3253         | 0.0935   | 0.0712 |
| 0.9195        | 11.67 | 4200  | 13.7718         | 0.1051   | 0.0793 |
| 0.9195        | 11.94 | 4300  | 12.9579         | 0.1070   | 0.0741 |
| 0.9195        | 12.22 | 4400  | 12.9222         | 0.1019   | 0.0727 |
| 0.8308        | 12.5  | 4500  | 12.8426         | 0.1058   | 0.0748 |
| 0.8308        | 12.78 | 4600  | 13.0151         | 0.1044   | 0.0800 |
| 0.8308        | 13.06 | 4700  | 12.5798         | 0.1100   | 0.0792 |
| 0.8308        | 13.33 | 4800  | 12.3432         | 0.1052   | 0.0758 |
| 0.8308        | 13.61 | 4900  | 12.8850         | 0.1050   | 0.0800 |
| 0.774         | 13.89 | 5000  | 12.5100         | 0.1075   | 0.0803 |
| 0.774         | 14.17 | 5100  | 12.8300         | 0.1031   | 0.0757 |
| 0.774         | 14.44 | 5200  | 12.6709         | 0.1051   | 0.0798 |
| 0.774         | 14.72 | 5300  | 13.2432         | 0.1065   | 0.0783 |
| 0.774         | 15.0  | 5400  | 13.1242         | 0.1080   | 0.0797 |
| 0.7241        | 15.28 | 5500  | 13.3100         | 0.1079   | 0.0815 |
| 0.7241        | 15.56 | 5600  | 12.2066         | 0.1059   | 0.0786 |
| 0.7241        | 15.83 | 5700  | 14.1967         | 0.1033   | 0.0848 |
| 0.7241        | 16.11 | 5800  | 13.3712         | 0.1075   | 0.0831 |
| 0.7241        | 16.39 | 5900  | 13.0712         | 0.1046   | 0.0807 |
| 0.6809        | 16.67 | 6000  | 13.3231         | 0.1055   | 0.0814 |
| 0.6809        | 16.94 | 6100  | 13.5326         | 0.1070   | 0.0812 |
| 0.6809        | 17.22 | 6200  | 13.7487         | 0.1051   | 0.0804 |
| 0.6809        | 17.5  | 6300  | 12.1853         | 0.1050   | 0.0812 |
| 0.6809        | 17.78 | 6400  | 12.3673         | 0.1022   | 0.0779 |
| 0.651         | 18.06 | 6500  | 13.1333         | 0.1022   | 0.0798 |
| 0.651         | 18.33 | 6600  | 13.3400         | 0.1044   | 0.0807 |
| 0.651         | 18.61 | 6700  | 12.7509         | 0.1038   | 0.0797 |
| 0.651         | 18.89 | 6800  | 12.6380         | 0.1045   | 0.0812 |
| 0.651         | 19.17 | 6900  | 13.1808         | 0.1076   | 0.0831 |
| 0.6254        | 19.44 | 7000  | 13.0347         | 0.1082   | 0.0851 |
| 0.6254        | 19.72 | 7100  | 12.6187         | 0.1031   | 0.0774 |
| 0.6254        | 20.0  | 7200  | 13.5187         | 0.1051   | 0.0801 |
| 0.6254        | 20.28 | 7300  | 13.0479         | 0.1082   | 0.0809 |
| 0.6254        | 20.56 | 7400  | 12.7203         | 0.1100   | 0.0793 |
| 0.5961        | 20.83 | 7500  | 12.6789         | 0.1043   | 0.0780 |
| 0.5961        | 21.11 | 7600  | 12.7525         | 0.1045   | 0.0807 |
| 0.5961        | 21.39 | 7700  | 13.4722         | 0.1060   | 0.0794 |
| 0.5961        | 21.67 | 7800  | 13.1929         | 0.1073   | 0.0821 |
| 0.5961        | 21.94 | 7900  | 12.8445         | 0.1041   | 0.0810 |
| 0.5787        | 22.22 | 8000  | 12.3704         | 0.1070   | 0.0807 |
| 0.5787        | 22.5  | 8100  | 13.1430         | 0.1084   | 0.0829 |
| 0.5787        | 22.78 | 8200  | 13.0495         | 0.1076   | 0.0820 |
| 0.5787        | 23.06 | 8300  | 13.1554         | 0.1064   | 0.0822 |
| 0.5787        | 23.33 | 8400  | 13.5845         | 0.1074   | 0.0814 |
| 0.5615        | 23.61 | 8500  | 12.8609         | 0.1100   | 0.0823 |
| 0.5615        | 23.89 | 8600  | 13.0800         | 0.1084   | 0.0810 |
| 0.5615        | 24.17 | 8700  | 12.8942         | 0.1068   | 0.0795 |
| 0.5615        | 24.44 | 8800  | 13.1842         | 0.1098   | 0.0834 |
| 0.5615        | 24.72 | 8900  | 13.3674         | 0.1062   | 0.0818 |
| 0.5483        | 25.0  | 9000  | 12.5537         | 0.1085   | 0.0809 |
| 0.5483        | 25.28 | 9100  | 12.5065         | 0.1094   | 0.0821 |
| 0.5483        | 25.56 | 9200  | 12.9911         | 0.1083   | 0.0842 |
| 0.5483        | 25.83 | 9300  | 12.8359         | 0.1092   | 0.0849 |
| 0.5483        | 26.11 | 9400  | 13.4473         | 0.1091   | 0.0840 |
| 0.5369        | 26.39 | 9500  | 12.6063         | 0.1083   | 0.0826 |
| 0.5369        | 26.67 | 9600  | 12.8707         | 0.1052   | 0.0777 |
| 0.5369        | 26.94 | 9700  | 13.1274         | 0.1088   | 0.0824 |
| 0.5369        | 27.22 | 9800  | 12.8088         | 0.1077   | 0.0815 |
| 0.5369        | 27.5  | 9900  | 13.0837         | 0.1080   | 0.0842 |
| 0.5264        | 27.78 | 10000 | 12.5880         | 0.1078   | 0.0813 |
| 0.5264        | 28.06 | 10100 | 12.8597         | 0.1087   | 0.0835 |
| 0.5264        | 28.33 | 10200 | 12.7099         | 0.1085   | 0.0827 |
| 0.5264        | 28.61 | 10300 | 12.9146         | 0.1070   | 0.0825 |
| 0.5264        | 28.89 | 10400 | 12.8773         | 0.1085   | 0.0831 |
| 0.5172        | 29.17 | 10500 | 12.6886         | 0.1073   | 0.0806 |
| 0.5172        | 29.44 | 10600 | 12.8463         | 0.1076   | 0.0813 |
| 0.5172        | 29.72 | 10700 | 12.7620         | 0.1082   | 0.0824 |
| 0.5172        | 30.0  | 10800 | 12.8207         | 0.1087   | 0.0816 |


### Framework versions

- Transformers 4.33.3
- Pytorch 2.1.1+cu121
- Datasets 2.14.5
- Tokenizers 0.13.3
