transformers pytorch roberta fill-mask generated_from_trainer base_model:DouglasPontes/2020-Q1-filtered_tweets autotrain_compatible endpoints_compatible region:us
---
base_model: DouglasPontes/2020-Q1-filtered_tweets
tags:
- generated_from_trainer
model-index:
- name: 2020-Q2-75p-filtered
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# 2020-Q2-75p-filtered

This model is a fine-tuned version of [DouglasPontes/2020-Q1-filtered_tweets](https://huggingface.co/DouglasPontes/2020-Q1-filtered_tweets) on an unknown dataset.
It achieves the following results on the evaluation set:
- Loss: 2.2312

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 4.1e-07
- train_batch_size: 16
- eval_batch_size: 16
- seed: 42
- optimizer: Adam with betas=(0.9,0.98) and epsilon=1e-08
- lr_scheduler_type: linear
- training_steps: 2400000

### Training results

| Training Loss | Epoch | Step    | Validation Loss |
|:-------------:|:-----:|:-------:|:---------------:|
| No log        | 0.02  | 8000    | 2.4865          |
| 2.6592        | 0.04  | 16000   | 2.4598          |
| 2.6592        | 0.07  | 24000   | 2.4472          |
| 2.6211        | 0.09  | 32000   | 2.4341          |
| 2.6211        | 0.11  | 40000   | 2.4223          |
| 2.6048        | 0.13  | 48000   | 2.4217          |
| 2.6048        | 0.16  | 56000   | 2.4184          |
| 2.5861        | 0.18  | 64000   | 2.4062          |
| 2.5861        | 0.2   | 72000   | 2.3919          |
| 2.5736        | 0.22  | 80000   | 2.3896          |
| 2.5736        | 0.25  | 88000   | 2.3951          |
| 2.5559        | 0.27  | 96000   | 2.3903          |
| 2.5559        | 0.29  | 104000  | 2.3836          |
| 2.5551        | 0.31  | 112000  | 2.3749          |
| 2.5551        | 0.34  | 120000  | 2.3794          |
| 2.5371        | 0.36  | 128000  | 2.3733          |
| 2.5371        | 0.38  | 136000  | 2.3703          |
| 2.5417        | 0.4   | 144000  | 2.3662          |
| 2.5417        | 0.43  | 152000  | 2.3728          |
| 2.5316        | 0.45  | 160000  | 2.3643          |
| 2.5316        | 0.47  | 168000  | 2.3568          |
| 2.5296        | 0.49  | 176000  | 2.3555          |
| 2.5296        | 0.52  | 184000  | 2.3506          |
| 2.5215        | 0.54  | 192000  | 2.3482          |
| 2.5215        | 0.56  | 200000  | 2.3514          |
| 2.5274        | 0.58  | 208000  | 2.3531          |
| 2.5274        | 0.61  | 216000  | 2.3463          |
| 2.5215        | 0.63  | 224000  | 2.3470          |
| 2.5215        | 0.65  | 232000  | 2.3407          |
| 2.5096        | 0.67  | 240000  | 2.3400          |
| 2.5096        | 0.7   | 248000  | 2.3402          |
| 2.5176        | 0.72  | 256000  | 2.3308          |
| 2.5176        | 0.74  | 264000  | 2.3342          |
| 2.5048        | 0.76  | 272000  | 2.3333          |
| 2.5048        | 0.79  | 280000  | 2.3288          |
| 2.4979        | 0.81  | 288000  | 2.3298          |
| 2.4979        | 0.83  | 296000  | 2.3237          |
| 2.4963        | 0.85  | 304000  | 2.3266          |
| 2.4963        | 0.88  | 312000  | 2.3197          |
| 2.4972        | 0.9   | 320000  | 2.3271          |
| 2.4972        | 0.92  | 328000  | 2.3275          |
| 2.4969        | 0.94  | 336000  | 2.3210          |
| 2.4969        | 0.97  | 344000  | 2.3222          |
| 2.4961        | 0.99  | 352000  | 2.3242          |
| 2.4961        | 1.01  | 360000  | 2.3155          |
| 2.49          | 1.03  | 368000  | 2.3175          |
| 2.49          | 1.06  | 376000  | 2.3076          |
| 2.4847        | 1.08  | 384000  | 2.3138          |
| 2.4847        | 1.1   | 392000  | 2.3183          |
| 2.4767        | 1.12  | 400000  | 2.3118          |
| 2.4767        | 1.15  | 408000  | 2.3152          |
| 2.4788        | 1.17  | 416000  | 2.3089          |
| 2.4788        | 1.19  | 424000  | 2.3051          |
| 2.4738        | 1.21  | 432000  | 2.3102          |
| 2.4738        | 1.24  | 440000  | 2.3069          |
| 2.4635        | 1.26  | 448000  | 2.3004          |
| 2.4635        | 1.28  | 456000  | 2.3066          |
| 2.4828        | 1.3   | 464000  | 2.3078          |
| 2.4828        | 1.32  | 472000  | 2.3072          |
| 2.4675        | 1.35  | 480000  | 2.3073          |
| 2.4675        | 1.37  | 488000  | 2.3014          |
| 2.4676        | 1.39  | 496000  | 2.2987          |
| 2.4676        | 1.41  | 504000  | 2.2988          |
| 2.4678        | 1.44  | 512000  | 2.2971          |
| 2.4678        | 1.46  | 520000  | 2.2969          |
| 2.4634        | 1.48  | 528000  | 2.2990          |
| 2.4634        | 1.5   | 536000  | 2.2869          |
| 2.4657        | 1.53  | 544000  | 2.2936          |
| 2.4657        | 1.55  | 552000  | 2.2915          |
| 2.4607        | 1.57  | 560000  | 2.2903          |
| 2.4607        | 1.59  | 568000  | 2.2934          |
| 2.4558        | 1.62  | 576000  | 2.2845          |
| 2.4558        | 1.64  | 584000  | 2.2897          |
| 2.4662        | 1.66  | 592000  | 2.2928          |
| 2.4662        | 1.68  | 600000  | 2.2861          |
| 2.4658        | 1.71  | 608000  | 2.2883          |
| 2.4658        | 1.73  | 616000  | 2.2878          |
| 2.4533        | 1.75  | 624000  | 2.2892          |
| 2.4533        | 1.77  | 632000  | 2.2886          |
| 2.4575        | 1.8   | 640000  | 2.2894          |
| 2.4575        | 1.82  | 648000  | 2.2871          |
| 2.4565        | 1.84  | 656000  | 2.2798          |
| 2.4565        | 1.86  | 664000  | 2.2877          |
| 2.4548        | 1.89  | 672000  | 2.2859          |
| 2.4548        | 1.91  | 680000  | 2.2787          |
| 2.4507        | 1.93  | 688000  | 2.2780          |
| 2.4507        | 1.95  | 696000  | 2.2826          |
| 2.4455        | 1.98  | 704000  | 2.2838          |
| 2.4455        | 2.0   | 712000  | 2.2764          |
| 2.4516        | 2.02  | 720000  | 2.2814          |
| 2.4516        | 2.04  | 728000  | 2.2807          |
| 2.445         | 2.07  | 736000  | 2.2740          |
| 2.445         | 2.09  | 744000  | 2.2780          |
| 2.4466        | 2.11  | 752000  | 2.2775          |
| 2.4466        | 2.13  | 760000  | 2.2783          |
| 2.4476        | 2.16  | 768000  | 2.2763          |
| 2.4476        | 2.18  | 776000  | 2.2737          |
| 2.4449        | 2.2   | 784000  | 2.2753          |
| 2.4449        | 2.22  | 792000  | 2.2762          |
| 2.4424        | 2.25  | 800000  | 2.2767          |
| 2.4424        | 2.27  | 808000  | 2.2702          |
| 2.4528        | 2.29  | 816000  | 2.2655          |
| 2.4528        | 2.31  | 824000  | 2.2727          |
| 2.4523        | 2.34  | 832000  | 2.2733          |
| 2.4523        | 2.36  | 840000  | 2.2654          |
| 2.4395        | 2.38  | 848000  | 2.2674          |
| 2.4395        | 2.4   | 856000  | 2.2754          |
| 2.434         | 2.43  | 864000  | 2.2722          |
| 2.434         | 2.45  | 872000  | 2.2666          |
| 2.4407        | 2.47  | 880000  | 2.2656          |
| 2.4407        | 2.49  | 888000  | 2.2654          |
| 2.4352        | 2.52  | 896000  | 2.2630          |
| 2.4352        | 2.54  | 904000  | 2.2662          |
| 2.4393        | 2.56  | 912000  | 2.2692          |
| 2.4393        | 2.58  | 920000  | 2.2558          |
| 2.4378        | 2.61  | 928000  | 2.2619          |
| 2.4378        | 2.63  | 936000  | 2.2614          |
| 2.4392        | 2.65  | 944000  | 2.2578          |
| 2.4392        | 2.67  | 952000  | 2.2672          |
| 2.437         | 2.69  | 960000  | 2.2598          |
| 2.437         | 2.72  | 968000  | 2.2633          |
| 2.4388        | 2.74  | 976000  | 2.2566          |
| 2.4388        | 2.76  | 984000  | 2.2551          |
| 2.4386        | 2.78  | 992000  | 2.2606          |
| 2.4386        | 2.81  | 1000000 | 2.2634          |
| 2.4402        | 2.83  | 1008000 | 2.2641          |
| 2.4402        | 2.85  | 1016000 | 2.2619          |
| 2.4442        | 2.87  | 1024000 | 2.2584          |
| 2.4442        | 2.9   | 1032000 | 2.2579          |
| 2.4327        | 2.92  | 1040000 | 2.2523          |
| 2.4327        | 2.94  | 1048000 | 2.2562          |
| 2.4289        | 2.96  | 1056000 | 2.2593          |
| 2.4289        | 2.99  | 1064000 | 2.2562          |
| 2.4319        | 3.01  | 1072000 | 2.2536          |
| 2.4319        | 3.03  | 1080000 | 2.2603          |
| 2.4174        | 3.05  | 1088000 | 2.2549          |
| 2.4174        | 3.08  | 1096000 | 2.2595          |
| 2.4155        | 3.1   | 1104000 | 2.2555          |
| 2.4155        | 3.12  | 1112000 | 2.2501          |
| 2.427         | 3.14  | 1120000 | 2.2528          |
| 2.427         | 3.17  | 1128000 | 2.2529          |
| 2.4222        | 3.19  | 1136000 | 2.2536          |
| 2.4222        | 3.21  | 1144000 | 2.2582          |
| 2.4232        | 3.23  | 1152000 | 2.2522          |
| 2.4232        | 3.26  | 1160000 | 2.2525          |
| 2.4252        | 3.28  | 1168000 | 2.2538          |
| 2.4252        | 3.3   | 1176000 | 2.2512          |
| 2.4209        | 3.32  | 1184000 | 2.2557          |
| 2.4209        | 3.35  | 1192000 | 2.2445          |
| 2.4243        | 3.37  | 1200000 | 2.2570          |
| 2.4243        | 3.39  | 1208000 | 2.2539          |
| 2.4278        | 3.41  | 1216000 | 2.2514          |
| 2.4278        | 3.44  | 1224000 | 2.2454          |
| 2.4286        | 3.46  | 1232000 | 2.2463          |
| 2.4286        | 3.48  | 1240000 | 2.2506          |
| 2.4274        | 3.5   | 1248000 | 2.2427          |
| 2.4274        | 3.53  | 1256000 | 2.2535          |
| 2.4201        | 3.55  | 1264000 | 2.2517          |
| 2.4201        | 3.57  | 1272000 | 2.2436          |
| 2.4233        | 3.59  | 1280000 | 2.2430          |
| 2.4233        | 3.62  | 1288000 | 2.2470          |
| 2.4183        | 3.64  | 1296000 | 2.2446          |
| 2.4183        | 3.66  | 1304000 | 2.2539          |
| 2.428         | 3.68  | 1312000 | 2.2492          |
| 2.428         | 3.71  | 1320000 | 2.2544          |
| 2.4206        | 3.73  | 1328000 | 2.2478          |
| 2.4206        | 3.75  | 1336000 | 2.2420          |
| 2.4287        | 3.77  | 1344000 | 2.2442          |
| 2.4287        | 3.8   | 1352000 | 2.2426          |
| 2.4297        | 3.82  | 1360000 | 2.2426          |
| 2.4297        | 3.84  | 1368000 | 2.2481          |
| 2.4185        | 3.86  | 1376000 | 2.2449          |
| 2.4185        | 3.89  | 1384000 | 2.2468          |
| 2.4217        | 3.91  | 1392000 | 2.2467          |
| 2.4217        | 3.93  | 1400000 | 2.2463          |
| 2.4144        | 3.95  | 1408000 | 2.2482          |
| 2.4144        | 3.97  | 1416000 | 2.2424          |
| 2.4175        | 4.0   | 1424000 | 2.2415          |
| 2.4175        | 4.02  | 1432000 | 2.2451          |
| 2.4169        | 4.04  | 1440000 | 2.2443          |
| 2.4169        | 4.06  | 1448000 | 2.2389          |
| 2.4142        | 4.09  | 1456000 | 2.2377          |
| 2.4142        | 4.11  | 1464000 | 2.2399          |
| 2.4122        | 4.13  | 1472000 | 2.2447          |
| 2.4122        | 4.15  | 1480000 | 2.2456          |
| 2.4166        | 4.18  | 1488000 | 2.2451          |
| 2.4166        | 4.2   | 1496000 | 2.2369          |
| 2.4165        | 4.22  | 1504000 | 2.2426          |
| 2.4165        | 4.24  | 1512000 | 2.2384          |
| 2.4204        | 4.27  | 1520000 | 2.2454          |
| 2.4204        | 4.29  | 1528000 | 2.2422          |
| 2.4192        | 4.31  | 1536000 | 2.2423          |
| 2.4192        | 4.33  | 1544000 | 2.2435          |
| 2.4167        | 4.36  | 1552000 | 2.2451          |
| 2.4167        | 4.38  | 1560000 | 2.2443          |
| 2.4124        | 4.4   | 1568000 | 2.2430          |
| 2.4124        | 4.42  | 1576000 | 2.2422          |
| 2.406         | 4.45  | 1584000 | 2.2357          |
| 2.406         | 4.47  | 1592000 | 2.2395          |
| 2.4166        | 4.49  | 1600000 | 2.2378          |
| 2.4166        | 4.51  | 1608000 | 2.2420          |
| 2.4144        | 4.54  | 1616000 | 2.2402          |
| 2.4144        | 4.56  | 1624000 | 2.2384          |
| 2.4219        | 4.58  | 1632000 | 2.2438          |
| 2.4219        | 4.6   | 1640000 | 2.2455          |
| 2.4061        | 4.63  | 1648000 | 2.2397          |
| 2.4061        | 4.65  | 1656000 | 2.2354          |
| 2.411         | 4.67  | 1664000 | 2.2393          |
| 2.411         | 4.69  | 1672000 | 2.2388          |
| 2.4125        | 4.72  | 1680000 | 2.2406          |
| 2.4125        | 4.74  | 1688000 | 2.2330          |
| 2.4092        | 4.76  | 1696000 | 2.2336          |
| 2.4092        | 4.78  | 1704000 | 2.2398          |
| 2.4078        | 4.81  | 1712000 | 2.2368          |
| 2.4078        | 4.83  | 1720000 | 2.2361          |
| 2.4185        | 4.85  | 1728000 | 2.2378          |
| 2.4185        | 4.87  | 1736000 | 2.2339          |
| 2.4088        | 4.9   | 1744000 | 2.2366          |
| 2.4088        | 4.92  | 1752000 | 2.2385          |
| 2.4095        | 4.94  | 1760000 | 2.2337          |
| 2.4095        | 4.96  | 1768000 | 2.2413          |
| 2.4078        | 4.99  | 1776000 | 2.2377          |
| 2.4078        | 5.01  | 1784000 | 2.2302          |
| 2.4073        | 5.03  | 1792000 | 2.2357          |
| 2.4073        | 5.05  | 1800000 | 2.2384          |
| 2.4073        | 5.08  | 1808000 | 2.2322          |
| 2.4073        | 5.1   | 1816000 | 2.2344          |
| 2.4043        | 5.12  | 1824000 | 2.2327          |
| 2.4043        | 5.14  | 1832000 | 2.2350          |
| 2.4082        | 5.17  | 1840000 | 2.2376          |
| 2.4082        | 5.19  | 1848000 | 2.2363          |
| 2.4073        | 5.21  | 1856000 | 2.2323          |
| 2.4073        | 5.23  | 1864000 | 2.2419          |
| 2.4148        | 5.26  | 1872000 | 2.2293          |
| 2.4148        | 5.28  | 1880000 | 2.2346          |
| 2.4098        | 5.3   | 1888000 | 2.2372          |
| 2.4098        | 5.32  | 1896000 | 2.2371          |
| 2.407         | 5.34  | 1904000 | 2.2397          |
| 2.407         | 5.37  | 1912000 | 2.2300          |
| 2.4108        | 5.39  | 1920000 | 2.2317          |
| 2.4108        | 5.41  | 1928000 | 2.2350          |
| 2.4168        | 5.43  | 1936000 | 2.2343          |
| 2.4168        | 5.46  | 1944000 | 2.2327          |
| 2.4113        | 5.48  | 1952000 | 2.2363          |
| 2.4113        | 5.5   | 1960000 | 2.2314          |
| 2.4131        | 5.52  | 1968000 | 2.2303          |
| 2.4131        | 5.55  | 1976000 | 2.2353          |
| 2.4129        | 5.57  | 1984000 | 2.2353          |
| 2.4129        | 5.59  | 1992000 | 2.2296          |
| 2.4129        | 5.61  | 2000000 | 2.2314          |
| 2.4129        | 5.64  | 2008000 | 2.2288          |
| 2.4045        | 5.66  | 2016000 | 2.2347          |
| 2.4045        | 5.68  | 2024000 | 2.2349          |
| 2.4089        | 5.7   | 2032000 | 2.2310          |
| 2.4089        | 5.73  | 2040000 | 2.2342          |
| 2.4091        | 5.75  | 2048000 | 2.2320          |
| 2.4091        | 5.77  | 2056000 | 2.2311          |
| 2.4137        | 5.79  | 2064000 | 2.2278          |
| 2.4137        | 5.82  | 2072000 | 2.2344          |
| 2.4063        | 5.84  | 2080000 | 2.2339          |
| 2.4063        | 5.86  | 2088000 | 2.2271          |
| 2.4046        | 5.88  | 2096000 | 2.2263          |
| 2.4046        | 5.91  | 2104000 | 2.2369          |
| 2.4105        | 5.93  | 2112000 | 2.2330          |
| 2.4105        | 5.95  | 2120000 | 2.2361          |
| 2.4045        | 5.97  | 2128000 | 2.2320          |
| 2.4045        | 6.0   | 2136000 | 2.2283          |
| 2.4093        | 6.02  | 2144000 | 2.2262          |
| 2.4093        | 6.04  | 2152000 | 2.2294          |
| 2.4109        | 6.06  | 2160000 | 2.2334          |
| 2.4109        | 6.09  | 2168000 | 2.2363          |
| 2.4061        | 6.11  | 2176000 | 2.2309          |
| 2.4061        | 6.13  | 2184000 | 2.2269          |
| 2.4007        | 6.15  | 2192000 | 2.2369          |
| 2.4007        | 6.18  | 2200000 | 2.2297          |
| 2.4034        | 6.2   | 2208000 | 2.2267          |
| 2.4034        | 6.22  | 2216000 | 2.2310          |
| 2.4049        | 6.24  | 2224000 | 2.2362          |
| 2.4049        | 6.27  | 2232000 | 2.2319          |
| 2.4052        | 6.29  | 2240000 | 2.2308          |
| 2.4052        | 6.31  | 2248000 | 2.2225          |
| 2.4102        | 6.33  | 2256000 | 2.2366          |
| 2.4102        | 6.36  | 2264000 | 2.2327          |
| 2.4046        | 6.38  | 2272000 | 2.2305          |
| 2.4046        | 6.4   | 2280000 | 2.2309          |
| 2.4066        | 6.42  | 2288000 | 2.2291          |
| 2.4066        | 6.45  | 2296000 | 2.2301          |
| 2.4041        | 6.47  | 2304000 | 2.2378          |
| 2.4041        | 6.49  | 2312000 | 2.2317          |
| 2.4081        | 6.51  | 2320000 | 2.2326          |
| 2.4081        | 6.54  | 2328000 | 2.2412          |
| 2.4147        | 6.56  | 2336000 | 2.2349          |
| 2.4147        | 6.58  | 2344000 | 2.2296          |
| 2.4105        | 6.6   | 2352000 | 2.2313          |
| 2.4105        | 6.62  | 2360000 | 2.2297          |
| 2.4096        | 6.65  | 2368000 | 2.2241          |
| 2.4096        | 6.67  | 2376000 | 2.2322          |
| 2.4089        | 6.69  | 2384000 | 2.2344          |
| 2.4089        | 6.71  | 2392000 | 2.2291          |
| 2.4048        | 6.74  | 2400000 | 2.2274          |


### Framework versions

- Transformers 4.35.0.dev0
- Pytorch 2.0.1+cu117
- Datasets 2.14.5
- Tokenizers 0.14.0
