transformers pytorch xlm-roberta generated_from_trainer dataset:massive base_model:FacebookAI/xlm-roberta-base license:mit endpoints_compatible region:us
---
license: mit
base_model: FacebookAI/xlm-roberta-base
tags:
- generated_from_trainer
datasets:
- massive
metrics:
- accuracy
- f1
model-index:
- name: scenario-KD-PR-MSV-EN-CL-D2_data-en-massive_all_1_1
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# scenario-KD-PR-MSV-EN-CL-D2_data-en-massive_all_1_1

This model is a fine-tuned version of [FacebookAI/xlm-roberta-base](https://huggingface.co/FacebookAI/xlm-roberta-base) on the massive dataset.
It achieves the following results on the evaluation set:
- Loss: 6.0402
- Accuracy: 0.4900
- F1: 0.4424

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 5e-05
- train_batch_size: 32
- eval_batch_size: 32
- seed: 666
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 30

### Training results

| Training Loss | Epoch | Step  | Validation Loss | Accuracy | F1     |
|:-------------:|:-----:|:-----:|:---------------:|:--------:|:------:|
| No log        | 0.28  | 100   | 7.5378          | 0.2054   | 0.0683 |
| No log        | 0.56  | 200   | 6.7584          | 0.3029   | 0.1679 |
| No log        | 0.83  | 300   | 6.1838          | 0.3605   | 0.2430 |
| No log        | 1.11  | 400   | 6.0327          | 0.3699   | 0.2692 |
| 4.8934        | 1.39  | 500   | 5.6667          | 0.4111   | 0.3002 |
| 4.8934        | 1.67  | 600   | 5.7790          | 0.4050   | 0.3124 |
| 4.8934        | 1.94  | 700   | 5.4854          | 0.4381   | 0.3466 |
| 4.8934        | 2.22  | 800   | 5.5038          | 0.4274   | 0.3551 |
| 4.8934        | 2.5   | 900   | 5.7379          | 0.4214   | 0.3570 |
| 2.2762        | 2.78  | 1000  | 5.6617          | 0.4315   | 0.3586 |
| 2.2762        | 3.06  | 1100  | 5.3737          | 0.4670   | 0.3688 |
| 2.2762        | 3.33  | 1200  | 5.8262          | 0.4357   | 0.3586 |
| 2.2762        | 3.61  | 1300  | 5.5787          | 0.4451   | 0.3870 |
| 2.2762        | 3.89  | 1400  | 5.8382          | 0.4307   | 0.3650 |
| 1.5016        | 4.17  | 1500  | 5.9036          | 0.4421   | 0.3738 |
| 1.5016        | 4.44  | 1600  | 5.9435          | 0.4366   | 0.3749 |
| 1.5016        | 4.72  | 1700  | 6.2289          | 0.4303   | 0.3617 |
| 1.5016        | 5.0   | 1800  | 6.2064          | 0.4294   | 0.3723 |
| 1.5016        | 5.28  | 1900  | 6.1007          | 0.4375   | 0.3655 |
| 1.0866        | 5.56  | 2000  | 5.9348          | 0.4489   | 0.3800 |
| 1.0866        | 5.83  | 2100  | 6.0184          | 0.4482   | 0.3840 |
| 1.0866        | 6.11  | 2200  | 6.4128          | 0.4329   | 0.3823 |
| 1.0866        | 6.39  | 2300  | 6.3238          | 0.4431   | 0.3917 |
| 1.0866        | 6.67  | 2400  | 6.2291          | 0.4482   | 0.3990 |
| 0.8677        | 6.94  | 2500  | 6.0659          | 0.4588   | 0.3970 |
| 0.8677        | 7.22  | 2600  | 6.4519          | 0.4397   | 0.3919 |
| 0.8677        | 7.5   | 2700  | 5.9209          | 0.4735   | 0.4074 |
| 0.8677        | 7.78  | 2800  | 6.8011          | 0.4311   | 0.3764 |
| 0.8677        | 8.06  | 2900  | 6.6258          | 0.4286   | 0.3745 |
| 0.6959        | 8.33  | 3000  | 6.0205          | 0.4598   | 0.3965 |
| 0.6959        | 8.61  | 3100  | 6.4682          | 0.4452   | 0.3991 |
| 0.6959        | 8.89  | 3200  | 6.2001          | 0.4563   | 0.3975 |
| 0.6959        | 9.17  | 3300  | 6.3492          | 0.4551   | 0.3971 |
| 0.6959        | 9.44  | 3400  | 6.2074          | 0.4576   | 0.4026 |
| 0.6083        | 9.72  | 3500  | 7.1146          | 0.4255   | 0.3813 |
| 0.6083        | 10.0  | 3600  | 6.5633          | 0.4487   | 0.4069 |
| 0.6083        | 10.28 | 3700  | 6.1010          | 0.4707   | 0.4088 |
| 0.6083        | 10.56 | 3800  | 6.2278          | 0.4727   | 0.4166 |
| 0.6083        | 10.83 | 3900  | 7.1772          | 0.4332   | 0.3886 |
| 0.5292        | 11.11 | 4000  | 7.0066          | 0.4387   | 0.3958 |
| 0.5292        | 11.39 | 4100  | 7.4419          | 0.4220   | 0.3945 |
| 0.5292        | 11.67 | 4200  | 6.5553          | 0.4653   | 0.4125 |
| 0.5292        | 11.94 | 4300  | 6.6278          | 0.4586   | 0.4135 |
| 0.5292        | 12.22 | 4400  | 6.2774          | 0.4692   | 0.4059 |
| 0.4804        | 12.5  | 4500  | 6.0512          | 0.4801   | 0.4246 |
| 0.4804        | 12.78 | 4600  | 6.3185          | 0.4701   | 0.4186 |
| 0.4804        | 13.06 | 4700  | 6.4401          | 0.4631   | 0.4108 |
| 0.4804        | 13.33 | 4800  | 6.2653          | 0.4663   | 0.4213 |
| 0.4804        | 13.61 | 4900  | 6.5306          | 0.4583   | 0.4106 |
| 0.4325        | 13.89 | 5000  | 6.7602          | 0.4476   | 0.3999 |
| 0.4325        | 14.17 | 5100  | 6.2450          | 0.4744   | 0.4098 |
| 0.4325        | 14.44 | 5200  | 6.4444          | 0.4675   | 0.4249 |
| 0.4325        | 14.72 | 5300  | 6.8074          | 0.4482   | 0.4130 |
| 0.4325        | 15.0  | 5400  | 6.7639          | 0.4562   | 0.4111 |
| 0.3985        | 15.28 | 5500  | 6.1950          | 0.4749   | 0.4198 |
| 0.3985        | 15.56 | 5600  | 5.9213          | 0.4885   | 0.4311 |
| 0.3985        | 15.83 | 5700  | 6.2779          | 0.4640   | 0.4228 |
| 0.3985        | 16.11 | 5800  | 6.2863          | 0.4764   | 0.4259 |
| 0.3985        | 16.39 | 5900  | 6.3712          | 0.4719   | 0.4206 |
| 0.3744        | 16.67 | 6000  | 6.1818          | 0.4763   | 0.4311 |
| 0.3744        | 16.94 | 6100  | 5.9495          | 0.4932   | 0.4295 |
| 0.3744        | 17.22 | 6200  | 6.2344          | 0.4758   | 0.4278 |
| 0.3744        | 17.5  | 6300  | 6.2192          | 0.4773   | 0.4254 |
| 0.3744        | 17.78 | 6400  | 6.0373          | 0.4825   | 0.4292 |
| 0.3523        | 18.06 | 6500  | 6.0578          | 0.4844   | 0.4343 |
| 0.3523        | 18.33 | 6600  | 6.4294          | 0.4701   | 0.4219 |
| 0.3523        | 18.61 | 6700  | 6.3761          | 0.4754   | 0.4259 |
| 0.3523        | 18.89 | 6800  | 6.1985          | 0.4768   | 0.4264 |
| 0.3523        | 19.17 | 6900  | 6.4615          | 0.4751   | 0.4220 |
| 0.3322        | 19.44 | 7000  | 6.1346          | 0.4851   | 0.4302 |
| 0.3322        | 19.72 | 7100  | 6.0352          | 0.4871   | 0.4282 |
| 0.3322        | 20.0  | 7200  | 6.1533          | 0.4844   | 0.4350 |
| 0.3322        | 20.28 | 7300  | 6.2422          | 0.4767   | 0.4290 |
| 0.3322        | 20.56 | 7400  | 6.6000          | 0.4664   | 0.4275 |
| 0.3219        | 20.83 | 7500  | 6.2801          | 0.4773   | 0.4322 |
| 0.3219        | 21.11 | 7600  | 6.2024          | 0.4792   | 0.4283 |
| 0.3219        | 21.39 | 7700  | 6.2759          | 0.4764   | 0.4309 |
| 0.3219        | 21.67 | 7800  | 6.5024          | 0.4656   | 0.4226 |
| 0.3219        | 21.94 | 7900  | 6.1907          | 0.4809   | 0.4306 |
| 0.3074        | 22.22 | 8000  | 6.0753          | 0.4850   | 0.4332 |
| 0.3074        | 22.5  | 8100  | 5.9962          | 0.4906   | 0.4332 |
| 0.3074        | 22.78 | 8200  | 6.0495          | 0.4910   | 0.4322 |
| 0.3074        | 23.06 | 8300  | 6.2469          | 0.4791   | 0.4329 |
| 0.3074        | 23.33 | 8400  | 6.0686          | 0.4897   | 0.4364 |
| 0.296         | 23.61 | 8500  | 5.9709          | 0.4914   | 0.4409 |
| 0.296         | 23.89 | 8600  | 6.0164          | 0.4923   | 0.4414 |
| 0.296         | 24.17 | 8700  | 6.0804          | 0.4870   | 0.4384 |
| 0.296         | 24.44 | 8800  | 6.0943          | 0.4869   | 0.4351 |
| 0.296         | 24.72 | 8900  | 6.1010          | 0.4857   | 0.4326 |
| 0.2918        | 25.0  | 9000  | 6.0140          | 0.4877   | 0.4396 |
| 0.2918        | 25.28 | 9100  | 6.1158          | 0.4879   | 0.4319 |
| 0.2918        | 25.56 | 9200  | 6.0549          | 0.4907   | 0.4386 |
| 0.2918        | 25.83 | 9300  | 6.0634          | 0.4862   | 0.4349 |
| 0.2918        | 26.11 | 9400  | 6.0695          | 0.4900   | 0.4356 |
| 0.2822        | 26.39 | 9500  | 6.0804          | 0.4919   | 0.4392 |
| 0.2822        | 26.67 | 9600  | 6.1051          | 0.4848   | 0.4371 |
| 0.2822        | 26.94 | 9700  | 5.9125          | 0.4951   | 0.4443 |
| 0.2822        | 27.22 | 9800  | 6.1057          | 0.4868   | 0.4372 |
| 0.2822        | 27.5  | 9900  | 6.0316          | 0.4906   | 0.4366 |
| 0.2786        | 27.78 | 10000 | 6.0800          | 0.4889   | 0.4395 |
| 0.2786        | 28.06 | 10100 | 6.0678          | 0.4894   | 0.4397 |
| 0.2786        | 28.33 | 10200 | 6.1626          | 0.4830   | 0.4345 |
| 0.2786        | 28.61 | 10300 | 6.0285          | 0.4892   | 0.4411 |
| 0.2786        | 28.89 | 10400 | 6.0053          | 0.4897   | 0.4396 |
| 0.2731        | 29.17 | 10500 | 6.0126          | 0.4910   | 0.4421 |
| 0.2731        | 29.44 | 10600 | 6.0908          | 0.4867   | 0.4371 |
| 0.2731        | 29.72 | 10700 | 6.0650          | 0.4903   | 0.4422 |
| 0.2731        | 30.0  | 10800 | 6.0402          | 0.4900   | 0.4424 |


### Framework versions

- Transformers 4.33.3
- Pytorch 2.1.1+cu121
- Datasets 2.14.5
- Tokenizers 0.13.3
