safetensors generated_from_trainer base_model:facebook/xlm-roberta-xl license:mit region:us
---
license: mit
base_model: facebook/xlm-roberta-xl
tags:
- generated_from_trainer
metrics:
- precision
- recall
- f1
- accuracy
model-index:
- name: xlm-roberta-xl-final-lora1
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# xlm-roberta-xl-final-lora1

This model is a fine-tuned version of [facebook/xlm-roberta-xl](https://huggingface.co/facebook/xlm-roberta-xl) on an unknown dataset.
It achieves the following results on the evaluation set:
- Loss: 1.5425
- Precision: 0.9311
- Recall: 0.9333
- F1: 0.9322
- Accuracy: 0.9410

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 0.0001
- train_batch_size: 16
- eval_batch_size: 16
- seed: 42
- distributed_type: multi-GPU
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 40
- num_epochs: 40
- mixed_precision_training: Native AMP
- label_smoothing_factor: 0.2

### Training results

| Training Loss | Epoch | Step  | Validation Loss | Precision | Recall | F1     | Accuracy |
|:-------------:|:-----:|:-----:|:---------------:|:---------:|:------:|:------:|:--------:|
| 2.6796        | 1.0   | 250   | 1.9566          | 0.7893    | 0.8311 | 0.8097 | 0.8425   |
| 1.7926        | 2.0   | 500   | 1.6808          | 0.8659    | 0.8790 | 0.8724 | 0.8947   |
| 1.617         | 3.0   | 750   | 1.6059          | 0.8892    | 0.9019 | 0.8955 | 0.9130   |
| 1.5343        | 4.0   | 1000  | 1.5724          | 0.9029    | 0.9063 | 0.9046 | 0.9197   |
| 1.4818        | 5.0   | 1250  | 1.5505          | 0.9110    | 0.9113 | 0.9112 | 0.9265   |
| 1.4513        | 6.0   | 1500  | 1.5435          | 0.9109    | 0.9183 | 0.9146 | 0.9290   |
| 1.431         | 7.0   | 1750  | 1.5367          | 0.9150    | 0.9210 | 0.9180 | 0.9314   |
| 1.4121        | 8.0   | 2000  | 1.5275          | 0.9227    | 0.9246 | 0.9237 | 0.9347   |
| 1.3999        | 9.0   | 2250  | 1.5298          | 0.9178    | 0.9225 | 0.9202 | 0.9321   |
| 1.3883        | 10.0  | 2500  | 1.5353          | 0.9165    | 0.9255 | 0.9210 | 0.9322   |
| 1.3755        | 11.0  | 2750  | 1.5442          | 0.9149    | 0.9240 | 0.9194 | 0.9310   |
| 1.3705        | 12.0  | 3000  | 1.5335          | 0.9201    | 0.9280 | 0.9240 | 0.9362   |
| 1.3661        | 13.0  | 3250  | 1.5345          | 0.9271    | 0.9270 | 0.9270 | 0.9359   |
| 1.3585        | 14.0  | 3500  | 1.5408          | 0.9172    | 0.9243 | 0.9207 | 0.9344   |
| 1.3535        | 15.0  | 3750  | 1.5323          | 0.9270    | 0.9285 | 0.9278 | 0.9381   |
| 1.3508        | 16.0  | 4000  | 1.5410          | 0.9236    | 0.9270 | 0.9253 | 0.9357   |
| 1.3477        | 17.0  | 4250  | 1.5343          | 0.9275    | 0.9285 | 0.9280 | 0.9390   |
| 1.3443        | 18.0  | 4500  | 1.5291          | 0.9314    | 0.9302 | 0.9308 | 0.9399   |
| 1.3407        | 19.0  | 4750  | 1.5381          | 0.9245    | 0.9280 | 0.9262 | 0.9373   |
| 1.3402        | 20.0  | 5000  | 1.5376          | 0.9257    | 0.9297 | 0.9277 | 0.9380   |
| 1.3385        | 21.0  | 5250  | 1.5365          | 0.9278    | 0.9302 | 0.9290 | 0.9393   |
| 1.3371        | 22.0  | 5500  | 1.5363          | 0.9297    | 0.9308 | 0.9302 | 0.9406   |
| 1.3382        | 23.0  | 5750  | 1.5343          | 0.9277    | 0.9310 | 0.9293 | 0.9396   |
| 1.3359        | 24.0  | 6000  | 1.5414          | 0.9268    | 0.9297 | 0.9282 | 0.9394   |
| 1.334         | 25.0  | 6250  | 1.5421          | 0.9298    | 0.9289 | 0.9293 | 0.9398   |
| 1.3334        | 26.0  | 6500  | 1.5404          | 0.9315    | 0.9328 | 0.9321 | 0.9409   |
| 1.3333        | 27.0  | 6750  | 1.5441          | 0.9285    | 0.9319 | 0.9302 | 0.9397   |
| 1.3324        | 28.0  | 7000  | 1.5459          | 0.9280    | 0.9300 | 0.9290 | 0.9385   |
| 1.3316        | 29.0  | 7250  | 1.5434          | 0.9311    | 0.9327 | 0.9319 | 0.9401   |
| 1.3313        | 30.0  | 7500  | 1.5366          | 0.9338    | 0.9353 | 0.9345 | 0.9422   |
| 1.3304        | 31.0  | 7750  | 1.5429          | 0.9316    | 0.9311 | 0.9314 | 0.9406   |
| 1.3299        | 32.0  | 8000  | 1.5374          | 0.9304    | 0.9337 | 0.9320 | 0.9417   |
| 1.3296        | 33.0  | 8250  | 1.5437          | 0.9305    | 0.9338 | 0.9321 | 0.9410   |
| 1.3297        | 34.0  | 8500  | 1.5405          | 0.9304    | 0.9340 | 0.9322 | 0.9416   |
| 1.3284        | 35.0  | 8750  | 1.5392          | 0.9294    | 0.9327 | 0.9310 | 0.9414   |
| 1.3281        | 36.0  | 9000  | 1.5397          | 0.9293    | 0.9324 | 0.9309 | 0.9410   |
| 1.3285        | 37.0  | 9250  | 1.5422          | 0.9311    | 0.9333 | 0.9322 | 0.9419   |
| 1.3279        | 38.0  | 9500  | 1.5431          | 0.9301    | 0.9333 | 0.9317 | 0.9411   |
| 1.3278        | 39.0  | 9750  | 1.5427          | 0.9306    | 0.9334 | 0.9320 | 0.9411   |
| 1.3279        | 40.0  | 10000 | 1.5425          | 0.9311    | 0.9333 | 0.9322 | 0.9410   |


### Framework versions

- Transformers 4.35.2
- Pytorch 2.0.1+cu118
- Datasets 2.15.0
- Tokenizers 0.15.0
