transformers pytorch roberta fill-mask generated_from_trainer base_model:cardiffnlp/twitter-roberta-base-2019-90m license:mit autotrain_compatible endpoints_compatible region:us
---
license: mit
base_model: cardiffnlp/twitter-roberta-base-2019-90m
tags:
- generated_from_trainer
model-index:
- name: 2020-Q1-25p-filtered
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# 2020-Q1-25p-filtered

This model is a fine-tuned version of [cardiffnlp/twitter-roberta-base-2019-90m](https://huggingface.co/cardiffnlp/twitter-roberta-base-2019-90m) on an unknown dataset.
It achieves the following results on the evaluation set:
- Loss: 2.2233

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 4.1e-07
- train_batch_size: 16
- eval_batch_size: 16
- seed: 42
- optimizer: Adam with betas=(0.9,0.98) and epsilon=1e-08
- lr_scheduler_type: linear
- training_steps: 2400000

### Training results

| Training Loss | Epoch | Step    | Validation Loss |
|:-------------:|:-----:|:-------:|:---------------:|
| No log        | 0.02  | 8000    | 2.5883          |
| 2.7561        | 0.04  | 16000   | 2.4958          |
| 2.7561        | 0.07  | 24000   | 2.4376          |
| 2.531         | 0.09  | 32000   | 2.4090          |
| 2.531         | 0.11  | 40000   | 2.3791          |
| 2.4627        | 0.13  | 48000   | 2.3666          |
| 2.4627        | 0.15  | 56000   | 2.3457          |
| 2.4252        | 0.17  | 64000   | 2.3380          |
| 2.4252        | 0.2   | 72000   | 2.3298          |
| 2.4061        | 0.22  | 80000   | 2.3253          |
| 2.4061        | 0.24  | 88000   | 2.3177          |
| 2.395         | 0.26  | 96000   | 2.3131          |
| 2.395         | 0.28  | 104000  | 2.3058          |
| 2.3843        | 0.31  | 112000  | 2.3010          |
| 2.3843        | 0.33  | 120000  | 2.2925          |
| 2.3738        | 0.35  | 128000  | 2.2916          |
| 2.3738        | 0.37  | 136000  | 2.2947          |
| 2.3686        | 0.39  | 144000  | 2.2835          |
| 2.3686        | 0.42  | 152000  | 2.2864          |
| 2.3615        | 0.44  | 160000  | 2.2834          |
| 2.3615        | 0.46  | 168000  | 2.2768          |
| 2.3515        | 0.48  | 176000  | 2.2803          |
| 2.3515        | 0.5   | 184000  | 2.2804          |
| 2.3508        | 0.52  | 192000  | 2.2754          |
| 2.3508        | 0.55  | 200000  | 2.2767          |
| 2.35          | 0.57  | 208000  | 2.2742          |
| 2.35          | 0.59  | 216000  | 2.2722          |
| 2.3385        | 0.61  | 224000  | 2.2661          |
| 2.3385        | 0.63  | 232000  | 2.2706          |
| 2.3393        | 0.66  | 240000  | 2.2633          |
| 2.3393        | 0.68  | 248000  | 2.2648          |
| 2.3392        | 0.7   | 256000  | 2.2656          |
| 2.3392        | 0.72  | 264000  | 2.2660          |
| 2.3336        | 0.74  | 272000  | 2.2657          |
| 2.3336        | 0.76  | 280000  | 2.2605          |
| 2.3324        | 0.79  | 288000  | 2.2615          |
| 2.3324        | 0.81  | 296000  | 2.2551          |
| 2.3312        | 0.83  | 304000  | 2.2581          |
| 2.3312        | 0.85  | 312000  | 2.2626          |
| 2.3352        | 0.87  | 320000  | 2.2576          |
| 2.3352        | 0.9   | 328000  | 2.2553          |
| 2.3287        | 0.92  | 336000  | 2.2591          |
| 2.3287        | 0.94  | 344000  | 2.2558          |
| 2.321         | 0.96  | 352000  | 2.2603          |
| 2.321         | 0.98  | 360000  | 2.2569          |
| 2.3278        | 1.01  | 368000  | 2.2544          |
| 2.3278        | 1.03  | 376000  | 2.2604          |
| 2.319         | 1.05  | 384000  | 2.2535          |
| 2.319         | 1.07  | 392000  | 2.2420          |
| 2.3151        | 1.09  | 400000  | 2.2583          |
| 2.3151        | 1.11  | 408000  | 2.2535          |
| 2.3144        | 1.14  | 416000  | 2.2582          |
| 2.3144        | 1.16  | 424000  | 2.2496          |
| 2.3191        | 1.18  | 432000  | 2.2532          |
| 2.3191        | 1.2   | 440000  | 2.2515          |
| 2.3168        | 1.22  | 448000  | 2.2501          |
| 2.3168        | 1.25  | 456000  | 2.2453          |
| 2.3156        | 1.27  | 464000  | 2.2445          |
| 2.3156        | 1.29  | 472000  | 2.2485          |
| 2.3178        | 1.31  | 480000  | 2.2493          |
| 2.3178        | 1.33  | 488000  | 2.2443          |
| 2.3113        | 1.35  | 496000  | 2.2493          |
| 2.3113        | 1.38  | 504000  | 2.2493          |
| 2.3116        | 1.4   | 512000  | 2.2483          |
| 2.3116        | 1.42  | 520000  | 2.2459          |
| 2.3166        | 1.44  | 528000  | 2.2481          |
| 2.3166        | 1.46  | 536000  | 2.2542          |
| 2.3158        | 1.49  | 544000  | 2.2443          |
| 2.3158        | 1.51  | 552000  | 2.2402          |
| 2.3148        | 1.53  | 560000  | 2.2449          |
| 2.3148        | 1.55  | 568000  | 2.2415          |
| 2.3145        | 1.57  | 576000  | 2.2471          |
| 2.3145        | 1.6   | 584000  | 2.2469          |
| 2.3119        | 1.62  | 592000  | 2.2445          |
| 2.3119        | 1.64  | 600000  | 2.2487          |
| 2.3045        | 1.66  | 608000  | 2.2456          |
| 2.3045        | 1.68  | 616000  | 2.2466          |
| 2.3046        | 1.7   | 624000  | 2.2357          |
| 2.3046        | 1.73  | 632000  | 2.2448          |
| 2.3083        | 1.75  | 640000  | 2.2381          |
| 2.3083        | 1.77  | 648000  | 2.2439          |
| 2.3065        | 1.79  | 656000  | 2.2402          |
| 2.3065        | 1.81  | 664000  | 2.2439          |
| 2.307         | 1.84  | 672000  | 2.2409          |
| 2.307         | 1.86  | 680000  | 2.2426          |
| 2.3026        | 1.88  | 688000  | 2.2387          |
| 2.3026        | 1.9   | 696000  | 2.2357          |
| 2.2949        | 1.92  | 704000  | 2.2379          |
| 2.2949        | 1.95  | 712000  | 2.2408          |
| 2.2951        | 1.97  | 720000  | 2.2432          |
| 2.2951        | 1.99  | 728000  | 2.2444          |
| 2.3011        | 2.01  | 736000  | 2.2382          |
| 2.3011        | 2.03  | 744000  | 2.2391          |
| 2.3017        | 2.05  | 752000  | 2.2363          |
| 2.3017        | 2.08  | 760000  | 2.2444          |
| 2.2978        | 2.1   | 768000  | 2.2370          |
| 2.2978        | 2.12  | 776000  | 2.2350          |
| 2.2961        | 2.14  | 784000  | 2.2347          |
| 2.2961        | 2.16  | 792000  | 2.2386          |
| 2.2968        | 2.19  | 800000  | 2.2322          |
| 2.2968        | 2.21  | 808000  | 2.2403          |
| 2.2962        | 2.23  | 816000  | 2.2347          |
| 2.2962        | 2.25  | 824000  | 2.2398          |
| 2.2984        | 2.27  | 832000  | 2.2358          |
| 2.2984        | 2.29  | 840000  | 2.2412          |
| 2.3029        | 2.32  | 848000  | 2.2386          |
| 2.3029        | 2.34  | 856000  | 2.2346          |
| 2.2985        | 2.36  | 864000  | 2.2323          |
| 2.2985        | 2.38  | 872000  | 2.2387          |
| 2.2922        | 2.4   | 880000  | 2.2303          |
| 2.2922        | 2.43  | 888000  | 2.2326          |
| 2.2967        | 2.45  | 896000  | 2.2422          |
| 2.2967        | 2.47  | 904000  | 2.2350          |
| 2.2917        | 2.49  | 912000  | 2.2299          |
| 2.2917        | 2.51  | 920000  | 2.2308          |
| 2.2912        | 2.54  | 928000  | 2.2345          |
| 2.2912        | 2.56  | 936000  | 2.2264          |
| 2.2887        | 2.58  | 944000  | 2.2361          |
| 2.2887        | 2.6   | 952000  | 2.2319          |
| 2.2956        | 2.62  | 960000  | 2.2340          |
| 2.2956        | 2.64  | 968000  | 2.2356          |
| 2.2927        | 2.67  | 976000  | 2.2366          |
| 2.2927        | 2.69  | 984000  | 2.2335          |
| 2.2872        | 2.71  | 992000  | 2.2330          |
| 2.2872        | 2.73  | 1000000 | 2.2251          |
| 2.2936        | 2.75  | 1008000 | 2.2327          |
| 2.2936        | 2.78  | 1016000 | 2.2326          |
| 2.2899        | 2.8   | 1024000 | 2.2307          |
| 2.2899        | 2.82  | 1032000 | 2.2291          |
| 2.2931        | 2.84  | 1040000 | 2.2285          |
| 2.2931        | 2.86  | 1048000 | 2.2327          |
| 2.3042        | 2.88  | 1056000 | 2.2367          |
| 2.3042        | 2.91  | 1064000 | 2.2345          |
| 2.2864        | 2.93  | 1072000 | 2.2267          |
| 2.2864        | 2.95  | 1080000 | 2.2343          |
| 2.2933        | 2.97  | 1088000 | 2.2354          |
| 2.2933        | 2.99  | 1096000 | 2.2260          |
| 2.2909        | 3.02  | 1104000 | 2.2341          |
| 2.2909        | 3.04  | 1112000 | 2.2266          |
| 2.2889        | 3.06  | 1120000 | 2.2253          |
| 2.2889        | 3.08  | 1128000 | 2.2255          |
| 2.292         | 3.1   | 1136000 | 2.2194          |
| 2.292         | 3.13  | 1144000 | 2.2319          |
| 2.282         | 3.15  | 1152000 | 2.2221          |
| 2.282         | 3.17  | 1160000 | 2.2273          |
| 2.2827        | 3.19  | 1168000 | 2.2296          |
| 2.2827        | 3.21  | 1176000 | 2.2332          |
| 2.2937        | 3.23  | 1184000 | 2.2302          |
| 2.2937        | 3.26  | 1192000 | 2.2262          |
| 2.2845        | 3.28  | 1200000 | 2.2318          |
| 2.2845        | 3.3   | 1208000 | 2.2291          |
| 2.284         | 3.32  | 1216000 | 2.2327          |
| 2.284         | 3.34  | 1224000 | 2.2308          |
| 2.2923        | 3.37  | 1232000 | 2.2264          |
| 2.2923        | 3.39  | 1240000 | 2.2390          |
| 2.2859        | 3.41  | 1248000 | 2.2310          |
| 2.2859        | 3.43  | 1256000 | 2.2287          |
| 2.2879        | 3.45  | 1264000 | 2.2284          |
| 2.2879        | 3.47  | 1272000 | 2.2228          |
| 2.292         | 3.5   | 1280000 | 2.2296          |
| 2.292         | 3.52  | 1288000 | 2.2329          |
| 2.2827        | 3.54  | 1296000 | 2.2263          |
| 2.2827        | 3.56  | 1304000 | 2.2324          |
| 2.2829        | 3.58  | 1312000 | 2.2232          |
| 2.2829        | 3.61  | 1320000 | 2.2273          |
| 2.2863        | 3.63  | 1328000 | 2.2296          |
| 2.2863        | 3.65  | 1336000 | 2.2294          |
| 2.2796        | 3.67  | 1344000 | 2.2283          |
| 2.2796        | 3.69  | 1352000 | 2.2280          |
| 2.2835        | 3.72  | 1360000 | 2.2264          |
| 2.2835        | 3.74  | 1368000 | 2.2224          |
| 2.2875        | 3.76  | 1376000 | 2.2219          |
| 2.2875        | 3.78  | 1384000 | 2.2243          |
| 2.2792        | 3.8   | 1392000 | 2.2320          |
| 2.2792        | 3.82  | 1400000 | 2.2273          |
| 2.2932        | 3.85  | 1408000 | 2.2257          |
| 2.2932        | 3.87  | 1416000 | 2.2360          |
| 2.2899        | 3.89  | 1424000 | 2.2277          |
| 2.2899        | 3.91  | 1432000 | 2.2275          |
| 2.2859        | 3.93  | 1440000 | 2.2287          |
| 2.2859        | 3.96  | 1448000 | 2.2211          |
| 2.2876        | 3.98  | 1456000 | 2.2236          |
| 2.2876        | 4.0   | 1464000 | 2.2288          |
| 2.2879        | 4.02  | 1472000 | 2.2226          |
| 2.2879        | 4.04  | 1480000 | 2.2242          |
| 2.282         | 4.06  | 1488000 | 2.2286          |
| 2.282         | 4.09  | 1496000 | 2.2210          |
| 2.2828        | 4.11  | 1504000 | 2.2304          |
| 2.2828        | 4.13  | 1512000 | 2.2310          |
| 2.2765        | 4.15  | 1520000 | 2.2295          |
| 2.2765        | 4.17  | 1528000 | 2.2276          |
| 2.2839        | 4.2   | 1536000 | 2.2260          |
| 2.2839        | 4.22  | 1544000 | 2.2255          |
| 2.2845        | 4.24  | 1552000 | 2.2200          |
| 2.2845        | 4.26  | 1560000 | 2.2228          |
| 2.2816        | 4.28  | 1568000 | 2.2322          |
| 2.2816        | 4.31  | 1576000 | 2.2250          |
| 2.2965        | 4.33  | 1584000 | 2.2242          |
| 2.2965        | 4.35  | 1592000 | 2.2295          |
| 2.2806        | 4.37  | 1600000 | 2.2198          |
| 2.2806        | 4.39  | 1608000 | 2.2301          |
| 2.2868        | 4.41  | 1616000 | 2.2309          |
| 2.2868        | 4.44  | 1624000 | 2.2270          |
| 2.2907        | 4.46  | 1632000 | 2.2291          |
| 2.2907        | 4.48  | 1640000 | 2.2269          |
| 2.2809        | 4.5   | 1648000 | 2.2261          |
| 2.2809        | 4.52  | 1656000 | 2.2318          |
| 2.2876        | 4.55  | 1664000 | 2.2252          |
| 2.2876        | 4.57  | 1672000 | 2.2248          |
| 2.2844        | 4.59  | 1680000 | 2.2223          |
| 2.2844        | 4.61  | 1688000 | 2.2250          |
| 2.2841        | 4.63  | 1696000 | 2.2278          |
| 2.2841        | 4.65  | 1704000 | 2.2226          |
| 2.2851        | 4.68  | 1712000 | 2.2274          |
| 2.2851        | 4.7   | 1720000 | 2.2247          |
| 2.2863        | 4.72  | 1728000 | 2.2239          |
| 2.2863        | 4.74  | 1736000 | 2.2227          |
| 2.2788        | 4.76  | 1744000 | 2.2234          |
| 2.2788        | 4.79  | 1752000 | 2.2293          |
| 2.2849        | 4.81  | 1760000 | 2.2199          |
| 2.2849        | 4.83  | 1768000 | 2.2309          |
| 2.2826        | 4.85  | 1776000 | 2.2235          |
| 2.2826        | 4.87  | 1784000 | 2.2292          |
| 2.2809        | 4.9   | 1792000 | 2.2248          |
| 2.2809        | 4.92  | 1800000 | 2.2187          |
| 2.2865        | 4.94  | 1808000 | 2.2331          |
| 2.2865        | 4.96  | 1816000 | 2.2244          |
| 2.2773        | 4.98  | 1824000 | 2.2246          |
| 2.2773        | 5.0   | 1832000 | 2.2315          |
| 2.2738        | 5.03  | 1840000 | 2.2319          |
| 2.2738        | 5.05  | 1848000 | 2.2258          |
| 2.2806        | 5.07  | 1856000 | 2.2241          |
| 2.2806        | 5.09  | 1864000 | 2.2228          |
| 2.2822        | 5.11  | 1872000 | 2.2218          |
| 2.2822        | 5.14  | 1880000 | 2.2276          |
| 2.2866        | 5.16  | 1888000 | 2.2233          |
| 2.2866        | 5.18  | 1896000 | 2.2266          |
| 2.2831        | 5.2   | 1904000 | 2.2231          |
| 2.2831        | 5.22  | 1912000 | 2.2241          |
| 2.2875        | 5.24  | 1920000 | 2.2263          |
| 2.2875        | 5.27  | 1928000 | 2.2234          |
| 2.2802        | 5.29  | 1936000 | 2.2269          |
| 2.2802        | 5.31  | 1944000 | 2.2253          |
| 2.2905        | 5.33  | 1952000 | 2.2191          |
| 2.2905        | 5.35  | 1960000 | 2.2217          |
| 2.282         | 5.38  | 1968000 | 2.2212          |
| 2.282         | 5.4   | 1976000 | 2.2213          |
| 2.2798        | 5.42  | 1984000 | 2.2218          |
| 2.2798        | 5.44  | 1992000 | 2.2222          |
| 2.2864        | 5.46  | 2000000 | 2.2212          |
| 2.2864        | 5.49  | 2008000 | 2.2282          |
| 2.2867        | 5.51  | 2016000 | 2.2304          |
| 2.2867        | 5.53  | 2024000 | 2.2222          |
| 2.2834        | 5.55  | 2032000 | 2.2285          |
| 2.2834        | 5.57  | 2040000 | 2.2230          |
| 2.2851        | 5.59  | 2048000 | 2.2237          |
| 2.2851        | 5.62  | 2056000 | 2.2283          |
| 2.2774        | 5.64  | 2064000 | 2.2232          |
| 2.2774        | 5.66  | 2072000 | 2.2282          |
| 2.277         | 5.68  | 2080000 | 2.2271          |
| 2.277         | 5.7   | 2088000 | 2.2256          |
| 2.2868        | 5.73  | 2096000 | 2.2252          |
| 2.2868        | 5.75  | 2104000 | 2.2285          |
| 2.2727        | 5.77  | 2112000 | 2.2251          |
| 2.2727        | 5.79  | 2120000 | 2.2239          |
| 2.2803        | 5.81  | 2128000 | 2.2287          |
| 2.2803        | 5.84  | 2136000 | 2.2274          |
| 2.2785        | 5.86  | 2144000 | 2.2227          |
| 2.2785        | 5.88  | 2152000 | 2.2267          |
| 2.2829        | 5.9   | 2160000 | 2.2251          |
| 2.2829        | 5.92  | 2168000 | 2.2228          |
| 2.2816        | 5.94  | 2176000 | 2.2235          |
| 2.2816        | 5.97  | 2184000 | 2.2289          |
| 2.283         | 5.99  | 2192000 | 2.2238          |
| 2.283         | 6.01  | 2200000 | 2.2245          |
| 2.2761        | 6.03  | 2208000 | 2.2297          |
| 2.2761        | 6.05  | 2216000 | 2.2300          |
| 2.2823        | 6.08  | 2224000 | 2.2268          |
| 2.2823        | 6.1   | 2232000 | 2.2252          |
| 2.2715        | 6.12  | 2240000 | 2.2240          |
| 2.2715        | 6.14  | 2248000 | 2.2233          |
| 2.2809        | 6.16  | 2256000 | 2.2238          |
| 2.2809        | 6.18  | 2264000 | 2.2204          |
| 2.2823        | 6.21  | 2272000 | 2.2218          |
| 2.2823        | 6.23  | 2280000 | 2.2295          |
| 2.2848        | 6.25  | 2288000 | 2.2298          |
| 2.2848        | 6.27  | 2296000 | 2.2299          |
| 2.2847        | 6.29  | 2304000 | 2.2246          |
| 2.2847        | 6.32  | 2312000 | 2.2230          |
| 2.2783        | 6.34  | 2320000 | 2.2260          |
| 2.2783        | 6.36  | 2328000 | 2.2176          |
| 2.2791        | 6.38  | 2336000 | 2.2211          |
| 2.2791        | 6.4   | 2344000 | 2.2262          |
| 2.2797        | 6.43  | 2352000 | 2.2293          |
| 2.2797        | 6.45  | 2360000 | 2.2219          |
| 2.2784        | 6.47  | 2368000 | 2.2249          |
| 2.2784        | 6.49  | 2376000 | 2.2216          |
| 2.271         | 6.51  | 2384000 | 2.2256          |
| 2.271         | 6.53  | 2392000 | 2.2296          |
| 2.2787        | 6.56  | 2400000 | 2.2275          |


### Framework versions

- Transformers 4.35.0.dev0
- Pytorch 2.0.1+cu117
- Datasets 2.14.5
- Tokenizers 0.14.0
