transformers pytorch roberta fill-mask generated_from_trainer base_model:DouglasPontes/2020-Q1-full_tweets autotrain_compatible endpoints_compatible region:us
---
base_model: DouglasPontes/2020-Q1-full_tweets
tags:
- generated_from_trainer
model-index:
- name: 2020-Q2-full_tweets
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# 2020-Q2-full_tweets

This model is a fine-tuned version of [DouglasPontes/2020-Q1-full_tweets](https://huggingface.co/DouglasPontes/2020-Q1-full_tweets) on an unknown dataset.
It achieves the following results on the evaluation set:
- Loss: 2.0061

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 4.1e-07
- train_batch_size: 16
- eval_batch_size: 16
- seed: 42
- optimizer: Adam with betas=(0.9,0.98) and epsilon=1e-08
- lr_scheduler_type: linear
- training_steps: 2400000

### Training results

| Training Loss | Epoch | Step    | Validation Loss |
|:-------------:|:-----:|:-------:|:---------------:|
| No log        | 0.01  | 8000    | 2.1043          |
| 2.2608        | 0.02  | 16000   | 2.0934          |
| 2.2608        | 0.03  | 24000   | 2.0862          |
| 2.2409        | 0.03  | 32000   | 2.0805          |
| 2.2409        | 0.04  | 40000   | 2.0793          |
| 2.2278        | 0.05  | 48000   | 2.0718          |
| 2.2278        | 0.06  | 56000   | 2.0753          |
| 2.2059        | 0.07  | 64000   | 2.0668          |
| 2.2059        | 0.08  | 72000   | 2.0657          |
| 2.1997        | 0.09  | 80000   | 2.0620          |
| 2.1997        | 0.1   | 88000   | 2.0553          |
| 2.1988        | 0.1   | 96000   | 2.0569          |
| 2.1988        | 0.11  | 104000  | 2.0525          |
| 2.1861        | 0.12  | 112000  | 2.0556          |
| 2.1861        | 0.13  | 120000  | 2.0493          |
| 2.1823        | 0.14  | 128000  | 2.0509          |
| 2.1823        | 0.15  | 136000  | 2.0461          |
| 2.1851        | 0.16  | 144000  | 2.0476          |
| 2.1851        | 0.17  | 152000  | 2.0450          |
| 2.1862        | 0.17  | 160000  | 2.0469          |
| 2.1862        | 0.18  | 168000  | 2.0442          |
| 2.1741        | 0.19  | 176000  | 2.0456          |
| 2.1741        | 0.2   | 184000  | 2.0442          |
| 2.181         | 0.21  | 192000  | 2.0402          |
| 2.181         | 0.22  | 200000  | 2.0423          |
| 2.1692        | 0.23  | 208000  | 2.0413          |
| 2.1692        | 0.24  | 216000  | 2.0448          |
| 2.1678        | 0.24  | 224000  | 2.0418          |
| 2.1678        | 0.25  | 232000  | 2.0417          |
| 2.1756        | 0.26  | 240000  | 2.0342          |
| 2.1756        | 0.27  | 248000  | 2.0377          |
| 2.1752        | 0.28  | 256000  | 2.0381          |
| 2.1752        | 0.29  | 264000  | 2.0354          |
| 2.1673        | 0.3   | 272000  | 2.0381          |
| 2.1673        | 0.31  | 280000  | 2.0375          |
| 2.1585        | 0.31  | 288000  | 2.0336          |
| 2.1585        | 0.32  | 296000  | 2.0344          |
| 2.1703        | 0.33  | 304000  | 2.0348          |
| 2.1703        | 0.34  | 312000  | 2.0330          |
| 2.1667        | 0.35  | 320000  | 2.0352          |
| 2.1667        | 0.36  | 328000  | 2.0359          |
| 2.1649        | 0.37  | 336000  | 2.0317          |
| 2.1649        | 0.38  | 344000  | 2.0314          |
| 2.1564        | 0.38  | 352000  | 2.0306          |
| 2.1564        | 0.39  | 360000  | 2.0299          |
| 2.161         | 0.4   | 368000  | 2.0317          |
| 2.161         | 0.41  | 376000  | 2.0325          |
| 2.1551        | 0.42  | 384000  | 2.0274          |
| 2.1551        | 0.43  | 392000  | 2.0282          |
| 2.1602        | 0.44  | 400000  | 2.0301          |
| 2.1602        | 0.45  | 408000  | 2.0303          |
| 2.1581        | 0.45  | 416000  | 2.0260          |
| 2.1581        | 0.46  | 424000  | 2.0248          |
| 2.1494        | 0.47  | 432000  | 2.0265          |
| 2.1494        | 0.48  | 440000  | 2.0247          |
| 2.1508        | 0.49  | 448000  | 2.0231          |
| 2.1508        | 0.5   | 456000  | 2.0276          |
| 2.153         | 0.51  | 464000  | 2.0276          |
| 2.153         | 0.51  | 472000  | 2.0242          |
| 2.1489        | 0.52  | 480000  | 2.0259          |
| 2.1489        | 0.53  | 488000  | 2.0257          |
| 2.1468        | 0.54  | 496000  | 2.0275          |
| 2.1468        | 0.55  | 504000  | 2.0303          |
| 2.1446        | 0.56  | 512000  | 2.0248          |
| 2.1446        | 0.57  | 520000  | 2.0286          |
| 2.1409        | 0.58  | 528000  | 2.0211          |
| 2.1409        | 0.58  | 536000  | 2.0204          |
| 2.1536        | 0.59  | 544000  | 2.0199          |
| 2.1536        | 0.6   | 552000  | 2.0281          |
| 2.1416        | 0.61  | 560000  | 2.0237          |
| 2.1416        | 0.62  | 568000  | 2.0231          |
| 2.1502        | 0.63  | 576000  | 2.0205          |
| 2.1502        | 0.64  | 584000  | 2.0217          |
| 2.1424        | 0.65  | 592000  | 2.0242          |
| 2.1424        | 0.65  | 600000  | 2.0238          |
| 2.1469        | 0.66  | 608000  | 2.0192          |
| 2.1469        | 0.67  | 616000  | 2.0249          |
| 2.145         | 0.68  | 624000  | 2.0196          |
| 2.145         | 0.69  | 632000  | 2.0224          |
| 2.1503        | 0.7   | 640000  | 2.0216          |
| 2.1503        | 0.71  | 648000  | 2.0228          |
| 2.1355        | 0.72  | 656000  | 2.0197          |
| 2.1355        | 0.72  | 664000  | 2.0240          |
| 2.1392        | 0.73  | 672000  | 2.0232          |
| 2.1392        | 0.74  | 680000  | 2.0209          |
| 2.1378        | 0.75  | 688000  | 2.0219          |
| 2.1378        | 0.76  | 696000  | 2.0192          |
| 2.1446        | 0.77  | 704000  | 2.0195          |
| 2.1446        | 0.78  | 712000  | 2.0197          |
| 2.1351        | 0.79  | 720000  | 2.0184          |
| 2.1351        | 0.79  | 728000  | 2.0162          |
| 2.1437        | 0.8   | 736000  | 2.0151          |
| 2.1437        | 0.81  | 744000  | 2.0202          |
| 2.1249        | 0.82  | 752000  | 2.0169          |
| 2.1249        | 0.83  | 760000  | 2.0189          |
| 2.1355        | 0.84  | 768000  | 2.0221          |
| 2.1355        | 0.85  | 776000  | 2.0194          |
| 2.1387        | 0.86  | 784000  | 2.0189          |
| 2.1387        | 0.86  | 792000  | 2.0165          |
| 2.1334        | 0.87  | 800000  | 2.0169          |
| 2.1334        | 0.88  | 808000  | 2.0189          |
| 2.137         | 0.89  | 816000  | 2.0162          |
| 2.137         | 0.9   | 824000  | 2.0168          |
| 2.1331        | 0.91  | 832000  | 2.0193          |
| 2.1331        | 0.92  | 840000  | 2.0166          |
| 2.1293        | 0.93  | 848000  | 2.0137          |
| 2.1293        | 0.93  | 856000  | 2.0183          |
| 2.1358        | 0.94  | 864000  | 2.0184          |
| 2.1358        | 0.95  | 872000  | 2.0171          |
| 2.1296        | 0.96  | 880000  | 2.0179          |
| 2.1296        | 0.97  | 888000  | 2.0152          |
| 2.1319        | 0.98  | 896000  | 2.0174          |
| 2.1319        | 0.99  | 904000  | 2.0206          |
| 2.1344        | 1.0   | 912000  | 2.0179          |
| 2.1344        | 1.0   | 920000  | 2.0154          |
| 2.1352        | 1.01  | 928000  | 2.0185          |
| 2.1352        | 1.02  | 936000  | 2.0170          |
| 2.1336        | 1.03  | 944000  | 2.0164          |
| 2.1336        | 1.04  | 952000  | 2.0137          |
| 2.1315        | 1.05  | 960000  | 2.0176          |
| 2.1315        | 1.06  | 968000  | 2.0155          |
| 2.1255        | 1.06  | 976000  | 2.0145          |
| 2.1255        | 1.07  | 984000  | 2.0233          |
| 2.1249        | 1.08  | 992000  | 2.0148          |
| 2.1249        | 1.09  | 1000000 | 2.0162          |
| 2.123         | 1.1   | 1008000 | 2.0174          |
| 2.123         | 1.11  | 1016000 | 2.0150          |
| 2.1263        | 1.12  | 1024000 | 2.0161          |
| 2.1263        | 1.13  | 1032000 | 2.0129          |
| 2.1232        | 1.13  | 1040000 | 2.0167          |
| 2.1232        | 1.14  | 1048000 | 2.0125          |
| 2.1168        | 1.15  | 1056000 | 2.0113          |
| 2.1168        | 1.16  | 1064000 | 2.0136          |
| 2.1307        | 1.17  | 1072000 | 2.0143          |
| 2.1307        | 1.18  | 1080000 | 2.0166          |
| 2.1336        | 1.19  | 1088000 | 2.0103          |
| 2.1336        | 1.2   | 1096000 | 2.0130          |
| 2.1227        | 1.2   | 1104000 | 2.0125          |
| 2.1227        | 1.21  | 1112000 | 2.0183          |
| 2.1223        | 1.22  | 1120000 | 2.0148          |
| 2.1223        | 1.23  | 1128000 | 2.0147          |
| 2.1289        | 1.24  | 1136000 | 2.0109          |
| 2.1289        | 1.25  | 1144000 | 2.0164          |
| 2.1278        | 1.26  | 1152000 | 2.0163          |
| 2.1278        | 1.27  | 1160000 | 2.0121          |
| 2.1261        | 1.27  | 1168000 | 2.0113          |
| 2.1261        | 1.28  | 1176000 | 2.0137          |
| 2.126         | 1.29  | 1184000 | 2.0152          |
| 2.126         | 1.3   | 1192000 | 2.0104          |
| 2.1235        | 1.31  | 1200000 | 2.0132          |
| 2.1235        | 1.32  | 1208000 | 2.0114          |
| 2.1229        | 1.33  | 1216000 | 2.0105          |
| 2.1229        | 1.34  | 1224000 | 2.0131          |
| 2.1213        | 1.34  | 1232000 | 2.0141          |
| 2.1213        | 1.35  | 1240000 | 2.0109          |
| 2.1185        | 1.36  | 1248000 | 2.0129          |
| 2.1185        | 1.37  | 1256000 | 2.0110          |
| 2.131         | 1.38  | 1264000 | 2.0123          |
| 2.131         | 1.39  | 1272000 | 2.0105          |
| 2.1141        | 1.4   | 1280000 | 2.0104          |
| 2.1141        | 1.41  | 1288000 | 2.0150          |
| 2.1219        | 1.41  | 1296000 | 2.0161          |
| 2.1219        | 1.42  | 1304000 | 2.0093          |
| 2.1203        | 1.43  | 1312000 | 2.0104          |
| 2.1203        | 1.44  | 1320000 | 2.0144          |
| 2.1264        | 1.45  | 1328000 | 2.0085          |
| 2.1264        | 1.46  | 1336000 | 2.0119          |
| 2.1194        | 1.47  | 1344000 | 2.0118          |
| 2.1194        | 1.48  | 1352000 | 2.0110          |
| 2.117         | 1.48  | 1360000 | 2.0147          |
| 2.117         | 1.49  | 1368000 | 2.0135          |
| 2.1311        | 1.5   | 1376000 | 2.0077          |
| 2.1311        | 1.51  | 1384000 | 2.0066          |
| 2.1215        | 1.52  | 1392000 | 2.0089          |
| 2.1215        | 1.53  | 1400000 | 2.0118          |
| 2.1185        | 1.54  | 1408000 | 2.0105          |
| 2.1185        | 1.54  | 1416000 | 2.0123          |
| 2.1284        | 1.55  | 1424000 | 2.0134          |
| 2.1284        | 1.56  | 1432000 | 2.0093          |
| 2.1174        | 1.57  | 1440000 | 2.0102          |
| 2.1174        | 1.58  | 1448000 | 2.0076          |
| 2.1108        | 1.59  | 1456000 | 2.0074          |
| 2.1108        | 1.6   | 1464000 | 2.0071          |
| 2.1252        | 1.61  | 1472000 | 2.0092          |
| 2.1252        | 1.61  | 1480000 | 2.0080          |
| 2.121         | 1.62  | 1488000 | 2.0053          |
| 2.121         | 1.63  | 1496000 | 2.0072          |
| 2.1178        | 1.64  | 1504000 | 2.0059          |
| 2.1178        | 1.65  | 1512000 | 2.0084          |
| 2.1154        | 1.66  | 1520000 | 2.0106          |
| 2.1154        | 1.67  | 1528000 | 2.0117          |
| 2.1214        | 1.68  | 1536000 | 2.0070          |
| 2.1214        | 1.68  | 1544000 | 2.0079          |
| 2.1175        | 1.69  | 1552000 | 2.0102          |
| 2.1175        | 1.7   | 1560000 | 2.0097          |
| 2.1206        | 1.71  | 1568000 | 2.0092          |
| 2.1206        | 1.72  | 1576000 | 2.0055          |
| 2.1302        | 1.73  | 1584000 | 2.0085          |
| 2.1302        | 1.74  | 1592000 | 2.0110          |
| 2.1177        | 1.75  | 1600000 | 2.0065          |
| 2.1177        | 1.75  | 1608000 | 2.0132          |
| 2.1101        | 1.76  | 1616000 | 2.0086          |
| 2.1101        | 1.77  | 1624000 | 2.0077          |
| 2.1194        | 1.78  | 1632000 | 2.0081          |
| 2.1194        | 1.79  | 1640000 | 2.0088          |
| 2.1167        | 1.8   | 1648000 | 2.0022          |
| 2.1167        | 1.81  | 1656000 | 2.0077          |
| 2.1083        | 1.82  | 1664000 | 2.0066          |
| 2.1083        | 1.82  | 1672000 | 2.0137          |
| 2.1232        | 1.83  | 1680000 | 2.0067          |
| 2.1232        | 1.84  | 1688000 | 2.0039          |
| 2.1212        | 1.85  | 1696000 | 2.0090          |
| 2.1212        | 1.86  | 1704000 | 2.0079          |
| 2.1246        | 1.87  | 1712000 | 2.0083          |
| 2.1246        | 1.88  | 1720000 | 2.0039          |
| 2.1129        | 1.89  | 1728000 | 2.0069          |
| 2.1129        | 1.89  | 1736000 | 2.0079          |
| 2.1209        | 1.9   | 1744000 | 2.0058          |
| 2.1209        | 1.91  | 1752000 | 2.0072          |
| 2.1209        | 1.92  | 1760000 | 2.0068          |
| 2.1209        | 1.93  | 1768000 | 2.0079          |
| 2.1184        | 1.94  | 1776000 | 2.0036          |
| 2.1184        | 1.95  | 1784000 | 2.0065          |
| 2.1065        | 1.96  | 1792000 | 2.0077          |
| 2.1065        | 1.96  | 1800000 | 2.0062          |
| 2.109         | 1.97  | 1808000 | 2.0090          |
| 2.109         | 1.98  | 1816000 | 2.0124          |
| 2.1081        | 1.99  | 1824000 | 2.0066          |
| 2.1081        | 2.0   | 1832000 | 2.0081          |
| 2.1151        | 2.01  | 1840000 | 2.0085          |
| 2.1151        | 2.02  | 1848000 | 2.0054          |
| 2.1178        | 2.03  | 1856000 | 2.0058          |
| 2.1178        | 2.03  | 1864000 | 2.0048          |
| 2.1035        | 2.04  | 1872000 | 2.0040          |
| 2.1035        | 2.05  | 1880000 | 2.0059          |
| 2.1197        | 2.06  | 1888000 | 2.0071          |
| 2.1197        | 2.07  | 1896000 | 2.0057          |
| 2.1143        | 2.08  | 1904000 | 2.0059          |
| 2.1143        | 2.09  | 1912000 | 2.0043          |
| 2.1082        | 2.09  | 1920000 | 2.0068          |
| 2.1082        | 2.1   | 1928000 | 2.0057          |
| 2.1202        | 2.11  | 1936000 | 2.0072          |
| 2.1202        | 2.12  | 1944000 | 2.0057          |
| 2.1138        | 2.13  | 1952000 | 2.0051          |
| 2.1138        | 2.14  | 1960000 | 2.0085          |
| 2.1082        | 2.15  | 1968000 | 2.0076          |
| 2.1082        | 2.16  | 1976000 | 2.0077          |
| 2.1084        | 2.16  | 1984000 | 2.0020          |
| 2.1084        | 2.17  | 1992000 | 2.0050          |
| 2.1151        | 2.18  | 2000000 | 2.0066          |
| 2.1151        | 2.19  | 2008000 | 2.0031          |
| 2.1141        | 2.2   | 2016000 | 2.0128          |
| 2.1141        | 2.21  | 2024000 | 2.0022          |
| 2.1129        | 2.22  | 2032000 | 2.0065          |
| 2.1129        | 2.23  | 2040000 | 2.0054          |
| 2.1164        | 2.23  | 2048000 | 2.0039          |
| 2.1164        | 2.24  | 2056000 | 2.0031          |
| 2.1121        | 2.25  | 2064000 | 2.0101          |
| 2.1121        | 2.26  | 2072000 | 2.0099          |
| 2.1071        | 2.27  | 2080000 | 2.0042          |
| 2.1071        | 2.28  | 2088000 | 2.0030          |
| 2.1094        | 2.29  | 2096000 | 2.0048          |
| 2.1094        | 2.3   | 2104000 | 2.0046          |
| 2.1017        | 2.3   | 2112000 | 2.0039          |
| 2.1017        | 2.31  | 2120000 | 2.0011          |
| 2.1124        | 2.32  | 2128000 | 2.0071          |
| 2.1124        | 2.33  | 2136000 | 2.0061          |
| 2.1064        | 2.34  | 2144000 | 2.0040          |
| 2.1064        | 2.35  | 2152000 | 2.0075          |
| 2.115         | 2.36  | 2160000 | 2.0026          |
| 2.115         | 2.37  | 2168000 | 2.0068          |
| 2.114         | 2.37  | 2176000 | 2.0066          |
| 2.114         | 2.38  | 2184000 | 2.0080          |
| 2.1171        | 2.39  | 2192000 | 2.0032          |
| 2.1171        | 2.4   | 2200000 | 2.0036          |
| 2.1119        | 2.41  | 2208000 | 2.0048          |
| 2.1119        | 2.42  | 2216000 | 2.0059          |
| 2.1097        | 2.43  | 2224000 | 2.0058          |
| 2.1097        | 2.44  | 2232000 | 2.0049          |
| 2.1091        | 2.44  | 2240000 | 2.0058          |
| 2.1091        | 2.45  | 2248000 | 2.0032          |
| 2.1107        | 2.46  | 2256000 | 2.0077          |
| 2.1107        | 2.47  | 2264000 | 2.0032          |
| 2.1126        | 2.48  | 2272000 | 2.0055          |
| 2.1126        | 2.49  | 2280000 | 2.0026          |
| 2.1173        | 2.5   | 2288000 | 2.0062          |
| 2.1173        | 2.51  | 2296000 | 2.0039          |
| 2.114         | 2.51  | 2304000 | 2.0064          |
| 2.114         | 2.52  | 2312000 | 2.0113          |
| 2.1131        | 2.53  | 2320000 | 2.0065          |
| 2.1131        | 2.54  | 2328000 | 2.0098          |
| 2.1045        | 2.55  | 2336000 | 2.0061          |
| 2.1045        | 2.56  | 2344000 | 2.0066          |
| 2.1144        | 2.57  | 2352000 | 2.0060          |
| 2.1144        | 2.57  | 2360000 | 2.0059          |
| 2.1086        | 2.58  | 2368000 | 2.0039          |
| 2.1086        | 2.59  | 2376000 | 2.0076          |
| 2.1058        | 2.6   | 2384000 | 2.0036          |
| 2.1058        | 2.61  | 2392000 | 2.0077          |
| 2.1112        | 2.62  | 2400000 | 2.0000          |


### Framework versions

- Transformers 4.35.0.dev0
- Pytorch 2.0.1+cu117
- Datasets 2.14.5
- Tokenizers 0.14.0
