transformers pytorch roberta fill-mask generated_from_trainer base_model:cardiffnlp/twitter-roberta-base-2019-90m license:mit autotrain_compatible endpoints_compatible region:us
---
license: mit
base_model: cardiffnlp/twitter-roberta-base-2019-90m
tags:
- generated_from_trainer
model-index:
- name: 2020-Q2-full_tweets_combined90
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# 2020-Q2-full_tweets_combined90

This model is a fine-tuned version of [cardiffnlp/twitter-roberta-base-2019-90m](https://huggingface.co/cardiffnlp/twitter-roberta-base-2019-90m) on an unknown dataset.
It achieves the following results on the evaluation set:
- Loss: 1.9422

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 4.1e-07
- train_batch_size: 16
- eval_batch_size: 16
- seed: 42
- optimizer: Adam with betas=(0.9,0.98) and epsilon=1e-08
- lr_scheduler_type: linear
- training_steps: 2400000

### Training results

| Training Loss | Epoch | Step    | Validation Loss |
|:-------------:|:-----:|:-------:|:---------------:|
| No log        | 0.02  | 8000    | 2.2220          |
| 2.4154        | 0.03  | 16000   | 2.1427          |
| 2.4154        | 0.05  | 24000   | 2.1028          |
| 2.2273        | 0.07  | 32000   | 2.0824          |
| 2.2273        | 0.08  | 40000   | 2.0645          |
| 2.1774        | 0.1   | 48000   | 2.0478          |
| 2.1774        | 0.12  | 56000   | 2.0327          |
| 2.1569        | 0.13  | 64000   | 2.0248          |
| 2.1569        | 0.15  | 72000   | 2.0209          |
| 2.1439        | 0.17  | 80000   | 2.0049          |
| 2.1439        | 0.19  | 88000   | 2.0113          |
| 2.1271        | 0.2   | 96000   | 2.0038          |
| 2.1271        | 0.22  | 104000  | 2.0065          |
| 2.1211        | 0.24  | 112000  | 1.9987          |
| 2.1211        | 0.25  | 120000  | 1.9929          |
| 2.1194        | 0.27  | 128000  | 1.9922          |
| 2.1194        | 0.29  | 136000  | 1.9917          |
| 2.1118        | 0.3   | 144000  | 1.9885          |
| 2.1118        | 0.32  | 152000  | 1.9870          |
| 2.1047        | 0.34  | 160000  | 1.9843          |
| 2.1047        | 0.35  | 168000  | 1.9827          |
| 2.1015        | 0.37  | 176000  | 1.9826          |
| 2.1015        | 0.39  | 184000  | 1.9774          |
| 2.1042        | 0.4   | 192000  | 1.9771          |
| 2.1042        | 0.42  | 200000  | 1.9770          |
| 2.0919        | 0.44  | 208000  | 1.9752          |
| 2.0919        | 0.45  | 216000  | 1.9775          |
| 2.0953        | 0.47  | 224000  | 1.9684          |
| 2.0953        | 0.49  | 232000  | 1.9748          |
| 2.0848        | 0.51  | 240000  | 1.9714          |
| 2.0848        | 0.52  | 248000  | 1.9781          |
| 2.0882        | 0.54  | 256000  | 1.9709          |
| 2.0882        | 0.56  | 264000  | 1.9660          |
| 2.0922        | 0.57  | 272000  | 1.9651          |
| 2.0922        | 0.59  | 280000  | 1.9678          |
| 2.0938        | 0.61  | 288000  | 1.9667          |
| 2.0938        | 0.62  | 296000  | 1.9630          |
| 2.095         | 0.64  | 304000  | 1.9642          |
| 2.095         | 0.66  | 312000  | 1.9624          |
| 2.0908        | 0.67  | 320000  | 1.9603          |
| 2.0908        | 0.69  | 328000  | 1.9649          |
| 2.0927        | 0.71  | 336000  | 1.9641          |
| 2.0927        | 0.72  | 344000  | 1.9603          |
| 2.0931        | 0.74  | 352000  | 1.9590          |
| 2.0931        | 0.76  | 360000  | 1.9644          |
| 2.087         | 0.77  | 368000  | 1.9635          |
| 2.087         | 0.79  | 376000  | 1.9614          |
| 2.0792        | 0.81  | 384000  | 1.9591          |
| 2.0792        | 0.83  | 392000  | 1.9575          |
| 2.0899        | 0.84  | 400000  | 1.9592          |
| 2.0899        | 0.86  | 408000  | 1.9619          |
| 2.0812        | 0.88  | 416000  | 1.9582          |
| 2.0812        | 0.89  | 424000  | 1.9580          |
| 2.0948        | 0.91  | 432000  | 1.9587          |
| 2.0948        | 0.93  | 440000  | 1.9593          |
| 2.0895        | 0.94  | 448000  | 1.9608          |
| 2.0895        | 0.96  | 456000  | 1.9566          |
| 2.0756        | 0.98  | 464000  | 1.9525          |
| 2.0756        | 0.99  | 472000  | 1.9541          |
| 2.0842        | 1.01  | 480000  | 1.9601          |
| 2.0842        | 1.03  | 488000  | 1.9564          |
| 2.0935        | 1.04  | 496000  | 1.9522          |
| 2.0935        | 1.06  | 504000  | 1.9532          |
| 2.0836        | 1.08  | 512000  | 1.9537          |
| 2.0836        | 1.09  | 520000  | 1.9553          |
| 2.0876        | 1.11  | 528000  | 1.9469          |
| 2.0876        | 1.13  | 536000  | 1.9497          |
| 2.0778        | 1.15  | 544000  | 1.9542          |
| 2.0778        | 1.16  | 552000  | 1.9516          |
| 2.0829        | 1.18  | 560000  | 1.9506          |
| 2.0829        | 1.2   | 568000  | 1.9505          |
| 2.0864        | 1.21  | 576000  | 1.9531          |
| 2.0864        | 1.23  | 584000  | 1.9455          |
| 2.0893        | 1.25  | 592000  | 1.9471          |
| 2.0893        | 1.26  | 600000  | 1.9539          |
| 2.0808        | 1.28  | 608000  | 1.9455          |
| 2.0808        | 1.3   | 616000  | 1.9497          |
| 2.0838        | 1.31  | 624000  | 1.9466          |
| 2.0838        | 1.33  | 632000  | 1.9498          |
| 2.0812        | 1.35  | 640000  | 1.9510          |
| 2.0812        | 1.36  | 648000  | 1.9526          |
| 2.0793        | 1.38  | 656000  | 1.9471          |
| 2.0793        | 1.4   | 664000  | 1.9469          |
| 2.0789        | 1.41  | 672000  | 1.9455          |
| 2.0789        | 1.43  | 680000  | 1.9469          |
| 2.0883        | 1.45  | 688000  | 1.9439          |
| 2.0883        | 1.47  | 696000  | 1.9439          |
| 2.09          | 1.48  | 704000  | 1.9416          |
| 2.09          | 1.5   | 712000  | 1.9492          |
| 2.0845        | 1.52  | 720000  | 1.9430          |
| 2.0845        | 1.53  | 728000  | 1.9484          |
| 2.0742        | 1.55  | 736000  | 1.9456          |
| 2.0742        | 1.57  | 744000  | 1.9380          |
| 2.0839        | 1.58  | 752000  | 1.9418          |
| 2.0839        | 1.6   | 760000  | 1.9434          |
| 2.0806        | 1.62  | 768000  | 1.9450          |
| 2.0806        | 1.63  | 776000  | 1.9426          |
| 2.0805        | 1.65  | 784000  | 1.9441          |
| 2.0805        | 1.67  | 792000  | 1.9459          |
| 2.0833        | 1.68  | 800000  | 1.9435          |
| 2.0833        | 1.7   | 808000  | 1.9455          |
| 2.0763        | 1.72  | 816000  | 1.9421          |
| 2.0763        | 1.73  | 824000  | 1.9438          |
| 2.0758        | 1.75  | 832000  | 1.9371          |
| 2.0758        | 1.77  | 840000  | 1.9432          |
| 2.0888        | 1.79  | 848000  | 1.9414          |
| 2.0888        | 1.8   | 856000  | 1.9444          |
| 2.0786        | 1.82  | 864000  | 1.9408          |
| 2.0786        | 1.84  | 872000  | 1.9397          |
| 2.079         | 1.85  | 880000  | 1.9406          |
| 2.079         | 1.87  | 888000  | 1.9442          |
| 2.0817        | 1.89  | 896000  | 1.9404          |
| 2.0817        | 1.9   | 904000  | 1.9450          |
| 2.0792        | 1.92  | 912000  | 1.9380          |
| 2.0792        | 1.94  | 920000  | 1.9385          |
| 2.0741        | 1.95  | 928000  | 1.9449          |
| 2.0741        | 1.97  | 936000  | 1.9414          |
| 2.0832        | 1.99  | 944000  | 1.9402          |
| 2.0832        | 2.0   | 952000  | 1.9410          |
| 2.0695        | 2.02  | 960000  | 1.9371          |
| 2.0695        | 2.04  | 968000  | 1.9342          |
| 2.0813        | 2.05  | 976000  | 1.9376          |
| 2.0813        | 2.07  | 984000  | 1.9397          |
| 2.0804        | 2.09  | 992000  | 1.9394          |
| 2.0804        | 2.11  | 1000000 | 1.9370          |
| 2.0789        | 2.12  | 1008000 | 1.9350          |
| 2.0789        | 2.14  | 1016000 | 1.9327          |
| 2.0754        | 2.16  | 1024000 | 1.9421          |
| 2.0754        | 2.17  | 1032000 | 1.9371          |
| 2.0774        | 2.19  | 1040000 | 1.9411          |
| 2.0774        | 2.21  | 1048000 | 1.9337          |
| 2.0766        | 2.22  | 1056000 | 1.9387          |
| 2.0766        | 2.24  | 1064000 | 1.9334          |
| 2.079         | 2.26  | 1072000 | 1.9386          |
| 2.079         | 2.27  | 1080000 | 1.9335          |
| 2.068         | 2.29  | 1088000 | 1.9363          |
| 2.068         | 2.31  | 1096000 | 1.9420          |
| 2.0786        | 2.32  | 1104000 | 1.9331          |
| 2.0786        | 2.34  | 1112000 | 1.9327          |
| 2.0734        | 2.36  | 1120000 | 1.9391          |
| 2.0734        | 2.37  | 1128000 | 1.9363          |
| 2.0787        | 2.39  | 1136000 | 1.9321          |
| 2.0787        | 2.41  | 1144000 | 1.9333          |
| 2.0731        | 2.43  | 1152000 | 1.9369          |
| 2.0731        | 2.44  | 1160000 | 1.9357          |
| 2.0816        | 2.46  | 1168000 | 1.9353          |
| 2.0816        | 2.48  | 1176000 | 1.9319          |
| 2.0758        | 2.49  | 1184000 | 1.9366          |
| 2.0758        | 2.51  | 1192000 | 1.9301          |
| 2.0725        | 2.53  | 1200000 | 1.9329          |
| 2.0725        | 2.54  | 1208000 | 1.9370          |
| 2.085         | 2.56  | 1216000 | 1.9251          |
| 2.085         | 2.58  | 1224000 | 1.9369          |
| 2.0809        | 2.59  | 1232000 | 1.9377          |
| 2.0809        | 2.61  | 1240000 | 1.9398          |
| 2.0742        | 2.63  | 1248000 | 1.9368          |
| 2.0742        | 2.64  | 1256000 | 1.9389          |
| 2.0743        | 2.66  | 1264000 | 1.9287          |
| 2.0743        | 2.68  | 1272000 | 1.9337          |
| 2.0822        | 2.69  | 1280000 | 1.9323          |
| 2.0822        | 2.71  | 1288000 | 1.9348          |
| 2.0845        | 2.73  | 1296000 | 1.9328          |
| 2.0845        | 2.75  | 1304000 | 1.9324          |
| 2.0706        | 2.76  | 1312000 | 1.9304          |
| 2.0706        | 2.78  | 1320000 | 1.9322          |
| 2.0813        | 2.8   | 1328000 | 1.9320          |
| 2.0813        | 2.81  | 1336000 | 1.9379          |
| 2.0768        | 2.83  | 1344000 | 1.9283          |
| 2.0768        | 2.85  | 1352000 | 1.9352          |
| 2.0776        | 2.86  | 1360000 | 1.9266          |
| 2.0776        | 2.88  | 1368000 | 1.9339          |
| 2.0776        | 2.9   | 1376000 | 1.9371          |
| 2.0776        | 2.91  | 1384000 | 1.9353          |
| 2.072         | 2.93  | 1392000 | 1.9290          |
| 2.072         | 2.95  | 1400000 | 1.9337          |
| 2.077         | 2.96  | 1408000 | 1.9318          |
| 2.077         | 2.98  | 1416000 | 1.9326          |
| 2.0777        | 3.0   | 1424000 | 1.9338          |
| 2.0777        | 3.01  | 1432000 | 1.9307          |
| 2.0846        | 3.03  | 1440000 | 1.9305          |
| 2.0846        | 3.05  | 1448000 | 1.9312          |
| 2.0744        | 3.07  | 1456000 | 1.9332          |
| 2.0744        | 3.08  | 1464000 | 1.9313          |
| 2.0767        | 3.1   | 1472000 | 1.9311          |
| 2.0767        | 3.12  | 1480000 | 1.9322          |
| 2.082         | 3.13  | 1488000 | 1.9362          |
| 2.082         | 3.15  | 1496000 | 1.9329          |
| 2.0774        | 3.17  | 1504000 | 1.9335          |
| 2.0774        | 3.18  | 1512000 | 1.9342          |
| 2.0793        | 3.2   | 1520000 | 1.9326          |
| 2.0793        | 3.22  | 1528000 | 1.9313          |
| 2.0834        | 3.23  | 1536000 | 1.9302          |
| 2.0834        | 3.25  | 1544000 | 1.9299          |
| 2.0698        | 3.27  | 1552000 | 1.9288          |
| 2.0698        | 3.28  | 1560000 | 1.9311          |
| 2.0721        | 3.3   | 1568000 | 1.9262          |
| 2.0721        | 3.32  | 1576000 | 1.9320          |
| 2.0742        | 3.33  | 1584000 | 1.9278          |
| 2.0742        | 3.35  | 1592000 | 1.9333          |
| 2.0774        | 3.37  | 1600000 | 1.9252          |
| 2.0774        | 3.39  | 1608000 | 1.9301          |
| 2.0766        | 3.4   | 1616000 | 1.9344          |
| 2.0766        | 3.42  | 1624000 | 1.9320          |
| 2.0702        | 3.44  | 1632000 | 1.9307          |
| 2.0702        | 3.45  | 1640000 | 1.9304          |
| 2.0772        | 3.47  | 1648000 | 1.9280          |
| 2.0772        | 3.49  | 1656000 | 1.9324          |
| 2.0757        | 3.5   | 1664000 | 1.9343          |
| 2.0757        | 3.52  | 1672000 | 1.9312          |
| 2.0747        | 3.54  | 1680000 | 1.9304          |
| 2.0747        | 3.55  | 1688000 | 1.9360          |
| 2.068         | 3.57  | 1696000 | 1.9297          |
| 2.068         | 3.59  | 1704000 | 1.9337          |
| 2.0825        | 3.6   | 1712000 | 1.9293          |
| 2.0825        | 3.62  | 1720000 | 1.9295          |
| 2.0811        | 3.64  | 1728000 | 1.9315          |
| 2.0811        | 3.65  | 1736000 | 1.9279          |
| 2.0844        | 3.67  | 1744000 | 1.9289          |
| 2.0844        | 3.69  | 1752000 | 1.9279          |
| 2.0827        | 3.71  | 1760000 | 1.9283          |
| 2.0827        | 3.72  | 1768000 | 1.9295          |
| 2.0684        | 3.74  | 1776000 | 1.9281          |
| 2.0684        | 3.76  | 1784000 | 1.9330          |
| 2.0724        | 3.77  | 1792000 | 1.9294          |
| 2.0724        | 3.79  | 1800000 | 1.9276          |
| 2.074         | 3.81  | 1808000 | 1.9227          |
| 2.074         | 3.82  | 1816000 | 1.9320          |
| 2.0801        | 3.84  | 1824000 | 1.9275          |
| 2.0801        | 3.86  | 1832000 | 1.9302          |
| 2.0783        | 3.87  | 1840000 | 1.9333          |
| 2.0783        | 3.89  | 1848000 | 1.9296          |
| 2.0787        | 3.91  | 1856000 | 1.9302          |
| 2.0787        | 3.92  | 1864000 | 1.9347          |
| 2.0733        | 3.94  | 1872000 | 1.9298          |
| 2.0733        | 3.96  | 1880000 | 1.9302          |
| 2.0742        | 3.97  | 1888000 | 1.9279          |
| 2.0742        | 3.99  | 1896000 | 1.9258          |
| 2.0769        | 4.01  | 1904000 | 1.9255          |
| 2.0769        | 4.03  | 1912000 | 1.9282          |
| 2.0736        | 4.04  | 1920000 | 1.9298          |
| 2.0736        | 4.06  | 1928000 | 1.9325          |
| 2.0713        | 4.08  | 1936000 | 1.9296          |
| 2.0713        | 4.09  | 1944000 | 1.9293          |
| 2.0825        | 4.11  | 1952000 | 1.9345          |
| 2.0825        | 4.13  | 1960000 | 1.9346          |
| 2.0828        | 4.14  | 1968000 | 1.9311          |
| 2.0828        | 4.16  | 1976000 | 1.9307          |
| 2.0821        | 4.18  | 1984000 | 1.9336          |
| 2.0821        | 4.19  | 1992000 | 1.9265          |
| 2.0768        | 4.21  | 2000000 | 1.9284          |
| 2.0768        | 4.23  | 2008000 | 1.9290          |
| 2.0695        | 4.24  | 2016000 | 1.9306          |
| 2.0695        | 4.26  | 2024000 | 1.9299          |
| 2.0698        | 4.28  | 2032000 | 1.9230          |
| 2.0698        | 4.29  | 2040000 | 1.9272          |
| 2.0776        | 4.31  | 2048000 | 1.9306          |
| 2.0776        | 4.33  | 2056000 | 1.9243          |
| 2.0797        | 4.35  | 2064000 | 1.9266          |
| 2.0797        | 4.36  | 2072000 | 1.9249          |
| 2.0808        | 4.38  | 2080000 | 1.9279          |
| 2.0808        | 4.4   | 2088000 | 1.9262          |
| 2.0776        | 4.41  | 2096000 | 1.9350          |
| 2.0776        | 4.43  | 2104000 | 1.9297          |
| 2.0805        | 4.45  | 2112000 | 1.9337          |
| 2.0805        | 4.46  | 2120000 | 1.9302          |
| 2.0791        | 4.48  | 2128000 | 1.9337          |
| 2.0791        | 4.5   | 2136000 | 1.9298          |
| 2.0771        | 4.51  | 2144000 | 1.9268          |
| 2.0771        | 4.53  | 2152000 | 1.9370          |
| 2.0807        | 4.55  | 2160000 | 1.9307          |
| 2.0807        | 4.56  | 2168000 | 1.9292          |
| 2.0856        | 4.58  | 2176000 | 1.9300          |
| 2.0856        | 4.6   | 2184000 | 1.9329          |
| 2.0744        | 4.61  | 2192000 | 1.9319          |
| 2.0744        | 4.63  | 2200000 | 1.9352          |
| 2.0839        | 4.65  | 2208000 | 1.9368          |
| 2.0839        | 4.67  | 2216000 | 1.9343          |
| 2.0706        | 4.68  | 2224000 | 1.9290          |
| 2.0706        | 4.7   | 2232000 | 1.9347          |
| 2.0745        | 4.72  | 2240000 | 1.9294          |
| 2.0745        | 4.73  | 2248000 | 1.9255          |
| 2.0767        | 4.75  | 2256000 | 1.9271          |
| 2.0767        | 4.77  | 2264000 | 1.9296          |
| 2.0753        | 4.78  | 2272000 | 1.9268          |
| 2.0753        | 4.8   | 2280000 | 1.9292          |
| 2.0716        | 4.82  | 2288000 | 1.9310          |
| 2.0716        | 4.83  | 2296000 | 1.9267          |
| 2.0778        | 4.85  | 2304000 | 1.9301          |
| 2.0778        | 4.87  | 2312000 | 1.9280          |
| 2.0724        | 4.88  | 2320000 | 1.9283          |
| 2.0724        | 4.9   | 2328000 | 1.9289          |
| 2.0811        | 4.92  | 2336000 | 1.9315          |
| 2.0811        | 4.93  | 2344000 | 1.9268          |
| 2.0816        | 4.95  | 2352000 | 1.9304          |
| 2.0816        | 4.97  | 2360000 | 1.9302          |
| 2.0775        | 4.99  | 2368000 | 1.9292          |
| 2.0775        | 5.0   | 2376000 | 1.9274          |
| 2.0807        | 5.02  | 2384000 | 1.9317          |
| 2.0807        | 5.04  | 2392000 | 1.9298          |
| 2.0668        | 5.05  | 2400000 | 1.9349          |


### Framework versions

- Transformers 4.35.0.dev0
- Pytorch 2.0.1+cu117
- Datasets 2.14.5
- Tokenizers 0.14.0
