transformers pytorch xlm-roberta generated_from_trainer dataset:massive base_model:FacebookAI/xlm-roberta-base license:mit endpoints_compatible region:us
---
license: mit
base_model: FacebookAI/xlm-roberta-base
tags:
- generated_from_trainer
datasets:
- massive
metrics:
- accuracy
- f1
model-index:
- name: scenario-KD-PR-MSV-EN-EN-D2_data-en-massive_all_1_1
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# scenario-KD-PR-MSV-EN-EN-D2_data-en-massive_all_1_1

This model is a fine-tuned version of [FacebookAI/xlm-roberta-base](https://huggingface.co/FacebookAI/xlm-roberta-base) on the massive dataset.
It achieves the following results on the evaluation set:
- Loss: 7.4557
- Accuracy: 0.4542
- F1: 0.3997

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 5e-05
- train_batch_size: 32
- eval_batch_size: 32
- seed: 47
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 30

### Training results

| Training Loss | Epoch | Step  | Validation Loss | Accuracy | F1     |
|:-------------:|:-----:|:-----:|:---------------:|:--------:|:------:|
| No log        | 0.28  | 100   | 7.5858          | 0.2343   | 0.0893 |
| No log        | 0.56  | 200   | 6.6990          | 0.3112   | 0.1823 |
| No log        | 0.83  | 300   | 6.0789          | 0.3867   | 0.2572 |
| No log        | 1.11  | 400   | 5.9605          | 0.4037   | 0.2779 |
| 5.219         | 1.39  | 500   | 5.9969          | 0.3801   | 0.2897 |
| 5.219         | 1.67  | 600   | 5.5708          | 0.4340   | 0.3063 |
| 5.219         | 1.94  | 700   | 5.5486          | 0.4323   | 0.3408 |
| 5.219         | 2.22  | 800   | 5.7112          | 0.4175   | 0.3328 |
| 5.219         | 2.5   | 900   | 5.6918          | 0.4241   | 0.3309 |
| 2.5566        | 2.78  | 1000  | 5.8073          | 0.4071   | 0.3317 |
| 2.5566        | 3.06  | 1100  | 6.4247          | 0.3718   | 0.3127 |
| 2.5566        | 3.33  | 1200  | 5.5318          | 0.4400   | 0.3548 |
| 2.5566        | 3.61  | 1300  | 6.1294          | 0.4157   | 0.3393 |
| 2.5566        | 3.89  | 1400  | 6.6057          | 0.3980   | 0.3468 |
| 1.7451        | 4.17  | 1500  | 5.7690          | 0.4434   | 0.3574 |
| 1.7451        | 4.44  | 1600  | 6.8472          | 0.3823   | 0.3434 |
| 1.7451        | 4.72  | 1700  | 6.0105          | 0.4302   | 0.3547 |
| 1.7451        | 5.0   | 1800  | 6.2133          | 0.4303   | 0.3651 |
| 1.7451        | 5.28  | 1900  | 6.1706          | 0.4450   | 0.3682 |
| 1.2832        | 5.56  | 2000  | 6.2779          | 0.4215   | 0.3666 |
| 1.2832        | 5.83  | 2100  | 6.2887          | 0.4386   | 0.3657 |
| 1.2832        | 6.11  | 2200  | 7.3324          | 0.3916   | 0.3450 |
| 1.2832        | 6.39  | 2300  | 6.5663          | 0.4371   | 0.3675 |
| 1.2832        | 6.67  | 2400  | 7.9241          | 0.3881   | 0.3289 |
| 0.9829        | 6.94  | 2500  | 7.1264          | 0.4101   | 0.3597 |
| 0.9829        | 7.22  | 2600  | 6.9262          | 0.4299   | 0.3637 |
| 0.9829        | 7.5   | 2700  | 7.3824          | 0.4249   | 0.3538 |
| 0.9829        | 7.78  | 2800  | 7.1468          | 0.4360   | 0.3630 |
| 0.9829        | 8.06  | 2900  | 8.1472          | 0.3922   | 0.3473 |
| 0.7736        | 8.33  | 3000  | 7.5301          | 0.4274   | 0.3677 |
| 0.7736        | 8.61  | 3100  | 8.5605          | 0.3840   | 0.3447 |
| 0.7736        | 8.89  | 3200  | 7.5599          | 0.4208   | 0.3663 |
| 0.7736        | 9.17  | 3300  | 8.0662          | 0.4022   | 0.3531 |
| 0.7736        | 9.44  | 3400  | 7.7026          | 0.4183   | 0.3572 |
| 0.6388        | 9.72  | 3500  | 7.3751          | 0.4351   | 0.3678 |
| 0.6388        | 10.0  | 3600  | 7.7079          | 0.4304   | 0.3679 |
| 0.6388        | 10.28 | 3700  | 8.1274          | 0.4168   | 0.3675 |
| 0.6388        | 10.56 | 3800  | 7.4026          | 0.4489   | 0.3834 |
| 0.6388        | 10.83 | 3900  | 7.8569          | 0.4347   | 0.3739 |
| 0.5491        | 11.11 | 4000  | 8.4369          | 0.4104   | 0.3712 |
| 0.5491        | 11.39 | 4100  | 7.6399          | 0.4294   | 0.3671 |
| 0.5491        | 11.67 | 4200  | 7.5023          | 0.4434   | 0.3744 |
| 0.5491        | 11.94 | 4300  | 7.7085          | 0.4315   | 0.3693 |
| 0.5491        | 12.22 | 4400  | 8.1275          | 0.4250   | 0.3695 |
| 0.4683        | 12.5  | 4500  | 7.7533          | 0.4468   | 0.3709 |
| 0.4683        | 12.78 | 4600  | 7.9580          | 0.4373   | 0.3787 |
| 0.4683        | 13.06 | 4700  | 7.7855          | 0.4377   | 0.3811 |
| 0.4683        | 13.33 | 4800  | 7.9553          | 0.4385   | 0.3789 |
| 0.4683        | 13.61 | 4900  | 7.5295          | 0.4490   | 0.3850 |
| 0.3993        | 13.89 | 5000  | 7.1115          | 0.4572   | 0.3868 |
| 0.3993        | 14.17 | 5100  | 8.5595          | 0.4124   | 0.3663 |
| 0.3993        | 14.44 | 5200  | 7.6619          | 0.4321   | 0.3723 |
| 0.3993        | 14.72 | 5300  | 7.8443          | 0.4255   | 0.3666 |
| 0.3993        | 15.0  | 5400  | 8.0150          | 0.4318   | 0.3822 |
| 0.3647        | 15.28 | 5500  | 8.4854          | 0.4144   | 0.3693 |
| 0.3647        | 15.56 | 5600  | 7.8110          | 0.4329   | 0.3757 |
| 0.3647        | 15.83 | 5700  | 7.7835          | 0.4451   | 0.3810 |
| 0.3647        | 16.11 | 5800  | 8.2197          | 0.4232   | 0.3743 |
| 0.3647        | 16.39 | 5900  | 8.0185          | 0.4334   | 0.3789 |
| 0.3214        | 16.67 | 6000  | 8.3538          | 0.4319   | 0.3835 |
| 0.3214        | 16.94 | 6100  | 7.8909          | 0.4354   | 0.3807 |
| 0.3214        | 17.22 | 6200  | 7.7767          | 0.4294   | 0.3834 |
| 0.3214        | 17.5  | 6300  | 8.1544          | 0.4283   | 0.3782 |
| 0.3214        | 17.78 | 6400  | 9.1679          | 0.4002   | 0.3678 |
| 0.3041        | 18.06 | 6500  | 8.5720          | 0.4133   | 0.3709 |
| 0.3041        | 18.33 | 6600  | 7.6984          | 0.4477   | 0.3926 |
| 0.3041        | 18.61 | 6700  | 7.5970          | 0.4499   | 0.3904 |
| 0.3041        | 18.89 | 6800  | 7.8427          | 0.4377   | 0.3773 |
| 0.3041        | 19.17 | 6900  | 7.4411          | 0.4497   | 0.3912 |
| 0.2783        | 19.44 | 7000  | 7.8410          | 0.4370   | 0.3820 |
| 0.2783        | 19.72 | 7100  | 7.8680          | 0.4463   | 0.3874 |
| 0.2783        | 20.0  | 7200  | 7.7858          | 0.4410   | 0.3729 |
| 0.2783        | 20.28 | 7300  | 7.0824          | 0.4589   | 0.3946 |
| 0.2783        | 20.56 | 7400  | 7.6646          | 0.4467   | 0.3884 |
| 0.261         | 20.83 | 7500  | 7.3004          | 0.4580   | 0.3888 |
| 0.261         | 21.11 | 7600  | 7.7126          | 0.4489   | 0.3863 |
| 0.261         | 21.39 | 7700  | 8.0802          | 0.4291   | 0.3828 |
| 0.261         | 21.67 | 7800  | 7.4386          | 0.4536   | 0.3931 |
| 0.261         | 21.94 | 7900  | 8.1070          | 0.4337   | 0.3846 |
| 0.2524        | 22.22 | 8000  | 7.4291          | 0.4540   | 0.3950 |
| 0.2524        | 22.5  | 8100  | 7.7144          | 0.4469   | 0.3891 |
| 0.2524        | 22.78 | 8200  | 8.2414          | 0.4295   | 0.3793 |
| 0.2524        | 23.06 | 8300  | 7.9833          | 0.4393   | 0.3864 |
| 0.2524        | 23.33 | 8400  | 7.3628          | 0.4541   | 0.3922 |
| 0.2414        | 23.61 | 8500  | 7.3701          | 0.4546   | 0.4025 |
| 0.2414        | 23.89 | 8600  | 7.7196          | 0.4451   | 0.3954 |
| 0.2414        | 24.17 | 8700  | 7.6399          | 0.4500   | 0.3948 |
| 0.2414        | 24.44 | 8800  | 7.4280          | 0.4543   | 0.3993 |
| 0.2414        | 24.72 | 8900  | 7.8374          | 0.4397   | 0.3915 |
| 0.2315        | 25.0  | 9000  | 7.4209          | 0.4561   | 0.3965 |
| 0.2315        | 25.28 | 9100  | 7.4840          | 0.4493   | 0.3969 |
| 0.2315        | 25.56 | 9200  | 7.5720          | 0.4492   | 0.3932 |
| 0.2315        | 25.83 | 9300  | 7.6239          | 0.4470   | 0.3952 |
| 0.2315        | 26.11 | 9400  | 7.9898          | 0.4362   | 0.3856 |
| 0.2238        | 26.39 | 9500  | 7.6913          | 0.4463   | 0.3955 |
| 0.2238        | 26.67 | 9600  | 7.5922          | 0.4494   | 0.3975 |
| 0.2238        | 26.94 | 9700  | 7.4909          | 0.4500   | 0.3979 |
| 0.2238        | 27.22 | 9800  | 7.5671          | 0.4492   | 0.3957 |
| 0.2238        | 27.5  | 9900  | 7.4782          | 0.4538   | 0.3964 |
| 0.2241        | 27.78 | 10000 | 7.4428          | 0.4550   | 0.4017 |
| 0.2241        | 28.06 | 10100 | 7.5369          | 0.4506   | 0.3962 |
| 0.2241        | 28.33 | 10200 | 7.5258          | 0.4479   | 0.3950 |
| 0.2241        | 28.61 | 10300 | 7.5294          | 0.4512   | 0.3934 |
| 0.2241        | 28.89 | 10400 | 7.2467          | 0.4606   | 0.4002 |
| 0.2167        | 29.17 | 10500 | 7.4440          | 0.4543   | 0.4012 |
| 0.2167        | 29.44 | 10600 | 7.6319          | 0.4470   | 0.3965 |
| 0.2167        | 29.72 | 10700 | 7.4810          | 0.4537   | 0.3996 |
| 0.2167        | 30.0  | 10800 | 7.4557          | 0.4542   | 0.3997 |


### Framework versions

- Transformers 4.33.3
- Pytorch 2.1.1+cu121
- Datasets 2.14.5
- Tokenizers 0.13.3
