
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# xlm-roberta-base-finetuned-panx-en

This model is a fine-tuned version of [xlm-roberta-base](https://huggingface.co/xlm-roberta-base) on the xtreme dataset.
It achieves the following results on the evaluation set:
- Loss: 0.2487
- F1: 0.8237
- Precision: 0.8163
- Recall: 0.8311
- Accuracy: 0.8237

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 5e-05
- train_batch_size: 24
- eval_batch_size: 24
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 3

### Training results

| Training Loss | Epoch | Step | Validation Loss | F1     | Precision | Recall | Accuracy |
|:-------------:|:-----:|:----:|:---------------:|:------:|:---------:|:------:|:--------:|
| 1.2644        | 0.03  | 24   | 0.8175          | 0.4212 | 0.3696    | 0.4897 | 0.4212   |
| 0.7209        | 0.06  | 48   | 0.5633          | 0.4817 | 0.4190    | 0.5665 | 0.4817   |
| 0.5951        | 0.09  | 72   | 0.4670          | 0.6059 | 0.5588    | 0.6617 | 0.6059   |
| 0.4475        | 0.12  | 96   | 0.4425          | 0.6659 | 0.6336    | 0.7016 | 0.6659   |
| 0.4978        | 0.14  | 120  | 0.4469          | 0.6375 | 0.5930    | 0.6892 | 0.6375   |
| 0.4383        | 0.17  | 144  | 0.4093          | 0.7003 | 0.6668    | 0.7374 | 0.7003   |
| 0.4148        | 0.2   | 168  | 0.3688          | 0.7122 | 0.6877    | 0.7387 | 0.7122   |
| 0.4513        | 0.23  | 192  | 0.3700          | 0.7236 | 0.7081    | 0.7397 | 0.7236   |
| 0.3786        | 0.26  | 216  | 0.3666          | 0.7304 | 0.7125    | 0.7493 | 0.7304   |
| 0.425         | 0.29  | 240  | 0.3652          | 0.7046 | 0.6874    | 0.7227 | 0.7046   |
| 0.4014        | 0.32  | 264  | 0.3438          | 0.7246 | 0.6964    | 0.7552 | 0.7246   |
| 0.3789        | 0.35  | 288  | 0.3533          | 0.7208 | 0.6922    | 0.7519 | 0.7208   |
| 0.4032        | 0.37  | 312  | 0.3567          | 0.7252 | 0.7125    | 0.7383 | 0.7252   |
| 0.371         | 0.4   | 336  | 0.3282          | 0.7433 | 0.7255    | 0.7620 | 0.7433   |
| 0.3397        | 0.43  | 360  | 0.3304          | 0.7522 | 0.7312    | 0.7745 | 0.7522   |
| 0.3871        | 0.46  | 384  | 0.3244          | 0.7427 | 0.7160    | 0.7715 | 0.7427   |
| 0.3461        | 0.49  | 408  | 0.3284          | 0.7520 | 0.7298    | 0.7756 | 0.7520   |
| 0.3504        | 0.52  | 432  | 0.3049          | 0.7574 | 0.7418    | 0.7737 | 0.7574   |
| 0.3387        | 0.55  | 456  | 0.3178          | 0.7717 | 0.7537    | 0.7906 | 0.7717   |
| 0.3259        | 0.58  | 480  | 0.3026          | 0.7738 | 0.7636    | 0.7843 | 0.7738   |
| 0.3473        | 0.6   | 504  | 0.3254          | 0.7324 | 0.7090    | 0.7574 | 0.7324   |
| 0.2893        | 0.63  | 528  | 0.3102          | 0.7689 | 0.7571    | 0.7810 | 0.7689   |
| 0.3669        | 0.66  | 552  | 0.3119          | 0.7631 | 0.7528    | 0.7737 | 0.7631   |
| 0.312         | 0.69  | 576  | 0.2963          | 0.7818 | 0.7734    | 0.7905 | 0.7818   |
| 0.297         | 0.72  | 600  | 0.3217          | 0.7542 | 0.7332    | 0.7765 | 0.7542   |
| 0.3095        | 0.75  | 624  | 0.3038          | 0.7732 | 0.7580    | 0.7891 | 0.7732   |
| 0.3514        | 0.78  | 648  | 0.2913          | 0.7794 | 0.7669    | 0.7924 | 0.7794   |
| 0.2824        | 0.81  | 672  | 0.3008          | 0.7813 | 0.7752    | 0.7876 | 0.7813   |
| 0.3203        | 0.83  | 696  | 0.2915          | 0.7807 | 0.7641    | 0.7980 | 0.7807   |
| 0.3089        | 0.86  | 720  | 0.2941          | 0.7838 | 0.7755    | 0.7923 | 0.7838   |
| 0.3174        | 0.89  | 744  | 0.2986          | 0.7770 | 0.7609    | 0.7937 | 0.7770   |
| 0.3264        | 0.92  | 768  | 0.2783          | 0.7788 | 0.7630    | 0.7951 | 0.7788   |
| 0.2815        | 0.95  | 792  | 0.2861          | 0.7848 | 0.7704    | 0.7998 | 0.7848   |
| 0.2895        | 0.98  | 816  | 0.2799          | 0.7842 | 0.7702    | 0.7988 | 0.7842   |
| 0.3023        | 1.01  | 840  | 0.2818          | 0.7876 | 0.7722    | 0.8038 | 0.7876   |
| 0.2358        | 1.04  | 864  | 0.2924          | 0.7836 | 0.7750    | 0.7925 | 0.7836   |
| 0.2819        | 1.06  | 888  | 0.2861          | 0.7761 | 0.7696    | 0.7828 | 0.7761   |
| 0.2692        | 1.09  | 912  | 0.2924          | 0.7756 | 0.7680    | 0.7833 | 0.7756   |
| 0.2478        | 1.12  | 936  | 0.2963          | 0.7833 | 0.7599    | 0.8082 | 0.7833   |
| 0.2557        | 1.15  | 960  | 0.2960          | 0.7783 | 0.7814    | 0.7751 | 0.7783   |
| 0.3003        | 1.18  | 984  | 0.2656          | 0.7862 | 0.7727    | 0.8002 | 0.7862   |
| 0.2254        | 1.21  | 1008 | 0.2791          | 0.8007 | 0.7890    | 0.8129 | 0.8007   |
| 0.2496        | 1.24  | 1032 | 0.2702          | 0.7877 | 0.7701    | 0.8062 | 0.7877   |
| 0.2124        | 1.27  | 1056 | 0.2888          | 0.7952 | 0.7895    | 0.8011 | 0.7952   |
| 0.2841        | 1.29  | 1080 | 0.2761          | 0.7946 | 0.7870    | 0.8023 | 0.7946   |
| 0.2517        | 1.32  | 1104 | 0.2659          | 0.8026 | 0.7909    | 0.8146 | 0.8026   |
| 0.2355        | 1.35  | 1128 | 0.2681          | 0.8003 | 0.7876    | 0.8134 | 0.8003   |
| 0.2402        | 1.38  | 1152 | 0.2701          | 0.7991 | 0.7892    | 0.8093 | 0.7991   |
| 0.2296        | 1.41  | 1176 | 0.2753          | 0.7946 | 0.7819    | 0.8077 | 0.7946   |
| 0.2453        | 1.44  | 1200 | 0.2696          | 0.8029 | 0.7912    | 0.8149 | 0.8029   |
| 0.2689        | 1.47  | 1224 | 0.2700          | 0.7936 | 0.7819    | 0.8056 | 0.7936   |
| 0.2362        | 1.5   | 1248 | 0.2705          | 0.8028 | 0.8005    | 0.8051 | 0.8028   |
| 0.226         | 1.53  | 1272 | 0.2642          | 0.8042 | 0.7910    | 0.8180 | 0.8042   |
| 0.2139        | 1.55  | 1296 | 0.2690          | 0.8013 | 0.7942    | 0.8084 | 0.8013   |
| 0.2744        | 1.58  | 1320 | 0.2619          | 0.7999 | 0.7841    | 0.8163 | 0.7999   |
| 0.2015        | 1.61  | 1344 | 0.2640          | 0.8066 | 0.8035    | 0.8098 | 0.8066   |
| 0.1949        | 1.64  | 1368 | 0.2750          | 0.8075 | 0.8023    | 0.8129 | 0.8075   |
| 0.2259        | 1.67  | 1392 | 0.2669          | 0.8092 | 0.7997    | 0.8189 | 0.8092   |
| 0.1884        | 1.7   | 1416 | 0.2729          | 0.8061 | 0.7990    | 0.8133 | 0.8061   |
| 0.1868        | 1.73  | 1440 | 0.2679          | 0.8083 | 0.8007    | 0.8161 | 0.8083   |
| 0.2292        | 1.76  | 1464 | 0.2658          | 0.8055 | 0.7954    | 0.8158 | 0.8055   |
| 0.22          | 1.78  | 1488 | 0.2610          | 0.8066 | 0.8006    | 0.8126 | 0.8066   |
| 0.2335        | 1.81  | 1512 | 0.2613          | 0.7997 | 0.7816    | 0.8185 | 0.7997   |
| 0.2379        | 1.84  | 1536 | 0.2495          | 0.8081 | 0.7975    | 0.8190 | 0.8081   |
| 0.2394        | 1.87  | 1560 | 0.2619          | 0.8063 | 0.7951    | 0.8177 | 0.8063   |
| 0.2526        | 1.9   | 1584 | 0.2502          | 0.8116 | 0.8032    | 0.8202 | 0.8116   |
| 0.2167        | 1.93  | 1608 | 0.2528          | 0.8134 | 0.8000    | 0.8273 | 0.8134   |
| 0.2354        | 1.96  | 1632 | 0.2449          | 0.8099 | 0.8013    | 0.8188 | 0.8099   |
| 0.2808        | 1.99  | 1656 | 0.2469          | 0.8067 | 0.7938    | 0.8201 | 0.8067   |
| 0.1924        | 2.01  | 1680 | 0.2487          | 0.8077 | 0.7930    | 0.8229 | 0.8077   |
| 0.1498        | 2.04  | 1704 | 0.2619          | 0.8127 | 0.8015    | 0.8242 | 0.8127   |
| 0.2           | 2.07  | 1728 | 0.2590          | 0.8133 | 0.8044    | 0.8224 | 0.8133   |
| 0.151         | 2.1   | 1752 | 0.2623          | 0.8066 | 0.7949    | 0.8186 | 0.8066   |
| 0.1646        | 2.13  | 1776 | 0.2632          | 0.8186 | 0.8137    | 0.8236 | 0.8186   |
| 0.1659        | 2.16  | 1800 | 0.2561          | 0.8188 | 0.8096    | 0.8281 | 0.8188   |
| 0.1888        | 2.19  | 1824 | 0.2549          | 0.8136 | 0.8038    | 0.8237 | 0.8136   |
| 0.2084        | 2.22  | 1848 | 0.2557          | 0.8141 | 0.8087    | 0.8197 | 0.8141   |
| 0.1571        | 2.24  | 1872 | 0.2697          | 0.8150 | 0.8053    | 0.8249 | 0.8150   |
| 0.1541        | 2.27  | 1896 | 0.2605          | 0.8191 | 0.8121    | 0.8262 | 0.8191   |
| 0.1586        | 2.3   | 1920 | 0.2742          | 0.8109 | 0.8073    | 0.8144 | 0.8109   |
| 0.1641        | 2.33  | 1944 | 0.2679          | 0.8148 | 0.8104    | 0.8193 | 0.8148   |
| 0.1914        | 2.36  | 1968 | 0.2596          | 0.8159 | 0.8056    | 0.8265 | 0.8159   |
| 0.1441        | 2.39  | 1992 | 0.2644          | 0.8183 | 0.8139    | 0.8226 | 0.8183   |
| 0.1672        | 2.42  | 2016 | 0.2652          | 0.8180 | 0.8081    | 0.8281 | 0.8180   |
| 0.1852        | 2.45  | 2040 | 0.2576          | 0.8205 | 0.8101    | 0.8313 | 0.8205   |
| 0.192         | 2.47  | 2064 | 0.2459          | 0.8179 | 0.8063    | 0.8298 | 0.8179   |
| 0.1698        | 2.5   | 2088 | 0.2482          | 0.8213 | 0.8149    | 0.8277 | 0.8213   |
| 0.1802        | 2.53  | 2112 | 0.2519          | 0.8155 | 0.8066    | 0.8247 | 0.8155   |
| 0.1619        | 2.56  | 2136 | 0.2582          | 0.8175 | 0.8036    | 0.8319 | 0.8175   |
| 0.1974        | 2.59  | 2160 | 0.2535          | 0.8184 | 0.8108    | 0.8261 | 0.8184   |
| 0.1655        | 2.62  | 2184 | 0.2514          | 0.8229 | 0.8165    | 0.8295 | 0.8229   |
| 0.1844        | 2.65  | 2208 | 0.2536          | 0.8208 | 0.8152    | 0.8264 | 0.8208   |
| 0.1601        | 2.68  | 2232 | 0.2531          | 0.8194 | 0.8104    | 0.8286 | 0.8194   |
| 0.161         | 2.71  | 2256 | 0.2508          | 0.8226 | 0.8145    | 0.8310 | 0.8226   |
| 0.1672        | 2.73  | 2280 | 0.2527          | 0.8216 | 0.8137    | 0.8296 | 0.8216   |
| 0.2053        | 2.76  | 2304 | 0.2482          | 0.8208 | 0.8112    | 0.8306 | 0.8208   |
| 0.1776        | 2.79  | 2328 | 0.2486          | 0.8215 | 0.8143    | 0.8288 | 0.8215   |
| 0.1559        | 2.82  | 2352 | 0.2495          | 0.8233 | 0.8156    | 0.8312 | 0.8233   |
| 0.1509        | 2.85  | 2376 | 0.2472          | 0.8231 | 0.8142    | 0.8322 | 0.8231   |
| 0.1695        | 2.88  | 2400 | 0.2465          | 0.8229 | 0.8134    | 0.8326 | 0.8229   |
| 0.1523        | 2.91  | 2424 | 0.2466          | 0.8234 | 0.8154    | 0.8315 | 0.8234   |
| 0.1525        | 2.94  | 2448 | 0.2478          | 0.8241 | 0.8165    | 0.8319 | 0.8241   |
| 0.1386        | 2.96  | 2472 | 0.2486          | 0.8236 | 0.8164    | 0.8309 | 0.8236   |
| 0.1532        | 2.99  | 2496 | 0.2487          | 0.8237 | 0.8163    | 0.8311 | 0.8237   |


### Framework versions

- Transformers 4.30.2
- Pytorch 2.0.1+cu118
- Datasets 2.13.0
- Tokenizers 0.13.3

{"license": "mit", "tags": ["generated_from_trainer"], "datasets": ["xtreme"], "metrics": ["f1", "precision", "recall", "accuracy"], "model-index": [{"name": "xlm-roberta-base-finetuned-panx-en", "results": [{"task": {"type": "token-classification", "name": "Token Classification"}, "dataset": {"name": "xtreme", "type": "xtreme", "config": "PAN-X.en", "split": "validation", "args": "PAN-X.en"}, "metrics": [{"type": "f1", "value": 0.8236654056326187, "name": "F1"}, {"type": "precision", "value": 0.8163449520899875, "name": "Precision"}, {"type": "recall", "value": 0.8311183373391772, "name": "Recall"}, {"type": "accuracy", "value": 0.8236654056326187, "name": "Accuracy"}]}]}]}
transformers pytorch tensorboard xlm-roberta token-classification generated_from_trainer dataset:xtreme license:mit model-index autotrain_compatible endpoints_compatible region:us
TAGS
#transformers #pytorch #tensorboard #xlm-roberta #token-classification #generated_from_trainer #dataset-xtreme #license-mit #model-index #autotrain_compatible #endpoints_compatible #region-us 

xlm-roberta-base-finetuned-panx-en
==================================


This model is a fine-tuned version of xlm-roberta-base on the xtreme dataset.
It achieves the following results on the evaluation set:


* Loss: 0.2487
* F1: 0.8237
* Precision: 0.8163
* Recall: 0.8311
* Accuracy: 0.8237


Model description
-----------------


More information needed


Intended uses & limitations
---------------------------


More information needed


Training and evaluation data
----------------------------


More information needed


Training procedure
------------------


### Training hyperparameters


The following hyperparameters were used during training:


* learning\_rate: 5e-05
* train\_batch\_size: 24
* eval\_batch\_size: 24
* seed: 42
* optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
* lr\_scheduler\_type: linear
* num\_epochs: 3


### Training results



### Framework versions


* Transformers 4.30.2
* Pytorch 2.0.1+cu118
* Datasets 2.13.0
* Tokenizers 0.13.3



### Training hyperparameters


The following hyperparameters were used during training:


* learning\_rate: 5e-05
* train\_batch\_size: 24
* eval\_batch\_size: 24
* seed: 42
* optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
* lr\_scheduler\_type: linear
* num\_epochs: 3 ### Training results ### Framework versions


* Transformers 4.30.2
* Pytorch 2.0.1+cu118
* Datasets 2.13.0
* Tokenizers 0.13.3
TAGS
#transformers #pytorch #tensorboard #xlm-roberta #token-classification #generated_from_trainer #dataset-xtreme #license-mit #model-index #autotrain_compatible #endpoints_compatible #region-us 
 ### Training hyperparameters


The following hyperparameters were used during training:


* learning\_rate: 5e-05
* train\_batch\_size: 24
* eval\_batch\_size: 24
* seed: 42
* optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
* lr\_scheduler\_type: linear
* num\_epochs: 3 ### Training results ### Framework versions


* Transformers 4.30.2
* Pytorch 2.0.1+cu118
* Datasets 2.13.0
* Tokenizers 0.13.3
TAGS
#transformers #pytorch #tensorboard #xlm-roberta #token-classification #generated_from_trainer #dataset-xtreme #license-mit #model-index #autotrain_compatible #endpoints_compatible #region-us 
### Training hyperparameters


The following hyperparameters were used during training:


* learning\_rate: 5e-05
* train\_batch\_size: 24
* eval\_batch\_size: 24
* seed: 42
* optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
* lr\_scheduler\_type: linear
* num\_epochs: 3### Training results### Framework versions


* Transformers 4.30.2
* Pytorch 2.0.1+cu118
* Datasets 2.13.0
* Tokenizers 0.13.3