2020-Q3-full_tweets

This model is a fine-tuned version of cardiffnlp/twitter-roberta-base-2019-90m on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 1.9157

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 4.1e-07
• train_batch_size: 32
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.98) and epsilon=1e-08
• lr_scheduler_type: linear
• training_steps: 2400000

Training results

Training Loss	Epoch	Step	Validation Loss
No log	0.03	8000	2.2197
2.3934	0.07	16000	2.1429
2.3934	0.1	24000	2.1001
2.2294	0.13	32000	2.0762
2.2294	0.17	40000	2.0515
2.1835	0.2	48000	2.0435
2.1835	0.24	56000	2.0346
2.1517	0.27	64000	2.0254
2.1517	0.3	72000	2.0175
2.1381	0.34	80000	2.0077
2.1381	0.37	88000	2.0029
2.1244	0.4	96000	2.0011
2.1244	0.44	104000	1.9980
2.1116	0.47	112000	1.9901
2.1116	0.51	120000	1.9840
2.1104	0.54	128000	1.9885
2.1104	0.57	136000	1.9855
2.1031	0.61	144000	1.9829
2.1031	0.64	152000	1.9813
2.0971	0.67	160000	1.9812
2.0971	0.71	168000	1.9795
2.1044	0.74	176000	1.9738
2.1044	0.77	184000	1.9768
2.0928	0.81	192000	1.9786
2.0928	0.84	200000	1.9699
2.0949	0.88	208000	1.9700
2.0949	0.91	216000	1.9653
2.0892	0.94	224000	1.9681
2.0892	0.98	232000	1.9650
2.0841	1.01	240000	1.9638
2.0841	1.04	248000	1.9682
2.0887	1.08	256000	1.9605
2.0887	1.11	264000	1.9614
2.0842	1.15	272000	1.9624
2.0842	1.18	280000	1.9605
2.0773	1.21	288000	1.9554
2.0773	1.25	296000	1.9578
2.0795	1.28	304000	1.9572
2.0795	1.31	312000	1.9521
2.0794	1.35	320000	1.9551
2.0794	1.38	328000	1.9569
2.0788	1.41	336000	1.9571
2.0788	1.45	344000	1.9502
2.0778	1.48	352000	1.9544
2.0778	1.52	360000	1.9470
2.0694	1.55	368000	1.9545
2.0694	1.58	376000	1.9472
2.0718	1.62	384000	1.9477
2.0718	1.65	392000	1.9496
2.0787	1.68	400000	1.9440
2.0787	1.72	408000	1.9484
2.0764	1.75	416000	1.9475
2.0764	1.79	424000	1.9469
2.0795	1.82	432000	1.9474
2.0795	1.85	440000	1.9492
2.07	1.89	448000	1.9480
2.07	1.92	456000	1.9482
2.0712	1.95	464000	1.9498
2.0712	1.99	472000	1.9429
2.0739	2.02	480000	1.9456
2.0739	2.05	488000	1.9469
2.0688	2.09	496000	1.9467
2.0688	2.12	504000	1.9454
2.0706	2.16	512000	1.9440
2.0706	2.19	520000	1.9401
2.0694	2.22	528000	1.9397
2.0694	2.26	536000	1.9429
2.0698	2.29	544000	1.9484
2.0698	2.32	552000	1.9375
2.0681	2.36	560000	1.9411
2.0681	2.39	568000	1.9419
2.0676	2.43	576000	1.9373
2.0676	2.46	584000	1.9366
2.0641	2.49	592000	1.9422
2.0641	2.53	600000	1.9365
2.0692	2.56	608000	1.9417
2.0692	2.59	616000	1.9385
2.0676	2.63	624000	1.9362
2.0676	2.66	632000	1.9414
2.0657	2.69	640000	1.9437
2.0657	2.73	648000	1.9356
2.0638	2.76	656000	1.9353
2.0638	2.8	664000	1.9385
2.0673	2.83	672000	1.9359
2.0673	2.86	680000	1.9314
2.0634	2.9	688000	1.9294
2.0634	2.93	696000	1.9346
2.0643	2.96	704000	1.9335
2.0643	3.0	712000	1.9316
2.0596	3.03	720000	1.9356
2.0596	3.07	728000	1.9390
2.0637	3.1	736000	1.9397
2.0637	3.13	744000	1.9375
2.0637	3.17	752000	1.9352
2.0637	3.2	760000	1.9310
2.0681	3.23	768000	1.9316
2.0681	3.27	776000	1.9269
2.0663	3.3	784000	1.9301
2.0663	3.33	792000	1.9354
2.0653	3.37	800000	1.9373
2.0653	3.4	808000	1.9354
2.0606	3.44	816000	1.9286
2.0606	3.47	824000	1.9318
2.0601	3.5	832000	1.9287
2.0601	3.54	840000	1.9280
2.0555	3.57	848000	1.9279
2.0555	3.6	856000	1.9297
2.0561	3.64	864000	1.9290
2.0561	3.67	872000	1.9252
2.066	3.71	880000	1.9274
2.066	3.74	888000	1.9257
2.0634	3.77	896000	1.9290
2.0634	3.81	904000	1.9267
2.0613	3.84	912000	1.9295
2.0613	3.87	920000	1.9300
2.0599	3.91	928000	1.9326
2.0599	3.94	936000	1.9313
2.0592	3.97	944000	1.9237
2.0592	4.01	952000	1.9272
2.0602	4.04	960000	1.9261
2.0602	4.08	968000	1.9283
2.0575	4.11	976000	1.9294
2.0575	4.14	984000	1.9284
2.0585	4.18	992000	1.9263
2.0585	4.21	1000000	1.9227
2.0535	4.24	1008000	1.9251
2.0535	4.28	1016000	1.9273
2.062	4.31	1024000	1.9242
2.062	4.35	1032000	1.9242
2.0606	4.38	1040000	1.9255
2.0606	4.41	1048000	1.9233
2.0565	4.45	1056000	1.9243
2.0565	4.48	1064000	1.9272
2.0538	4.51	1072000	1.9308
2.0538	4.55	1080000	1.9236
2.0573	4.58	1088000	1.9246
2.0573	4.61	1096000	1.9237
2.0562	4.65	1104000	1.9199
2.0562	4.68	1112000	1.9235
2.0534	4.72	1120000	1.9209
2.0534	4.75	1128000	1.9215
2.0567	4.78	1136000	1.9242
2.0567	4.82	1144000	1.9272
2.0592	4.85	1152000	1.9257
2.0592	4.88	1160000	1.9228
2.0599	4.92	1168000	1.9205
2.0599	4.95	1176000	1.9190
2.0504	4.99	1184000	1.9241
2.0504	5.02	1192000	1.9214
2.0541	5.05	1200000	1.9265
2.0541	5.09	1208000	1.9250
2.0581	5.12	1216000	1.9174
2.0581	5.15	1224000	1.9232
2.057	5.19	1232000	1.9242
2.057	5.22	1240000	1.9201
2.0541	5.25	1248000	1.9187
2.0541	5.29	1256000	1.9205
2.0542	5.32	1264000	1.9178
2.0542	5.36	1272000	1.9239
2.0526	5.39	1280000	1.9185
2.0526	5.42	1288000	1.9227
2.0503	5.46	1296000	1.9223
2.0503	5.49	1304000	1.9230
2.0579	5.52	1312000	1.9143
2.0579	5.56	1320000	1.9188
2.0523	5.59	1328000	1.9170
2.0523	5.63	1336000	1.9252
2.056	5.66	1344000	1.9183
2.056	5.69	1352000	1.9237
2.0545	5.73	1360000	1.9198
2.0545	5.76	1368000	1.9225
2.0552	5.79	1376000	1.9172
2.0552	5.83	1384000	1.9179
2.0571	5.86	1392000	1.9238
2.0571	5.89	1400000	1.9189
2.0637	5.93	1408000	1.9217
2.0637	5.96	1416000	1.9190
2.0554	6.0	1424000	1.9259
2.0554	6.03	1432000	1.9184
2.0545	6.06	1440000	1.9244
2.0545	6.1	1448000	1.9201
2.0538	6.13	1456000	1.9251
2.0538	6.16	1464000	1.9216
2.058	6.2	1472000	1.9221
2.058	6.23	1480000	1.9247
2.0482	6.27	1488000	1.9209
2.0482	6.3	1496000	1.9207
2.0528	6.33	1504000	1.9177
2.0528	6.37	1512000	1.9141
2.0529	6.4	1520000	1.9213
2.0529	6.43	1528000	1.9170
2.059	6.47	1536000	1.9161
2.059	6.5	1544000	1.9164
2.056	6.53	1552000	1.9177
2.056	6.57	1560000	1.9189
2.058	6.6	1568000	1.9181
2.058	6.64	1576000	1.9214
2.0543	6.67	1584000	1.9137
2.0543	6.7	1592000	1.9181
2.0513	6.74	1600000	1.9187
2.0513	6.77	1608000	1.9176
2.0587	6.8	1616000	1.9145
2.0587	6.84	1624000	1.9192
2.053	6.87	1632000	1.9202
2.053	6.91	1640000	1.9183
2.0543	6.94	1648000	1.9163
2.0543	6.97	1656000	1.9171
2.0492	7.01	1664000	1.9183
2.0492	7.04	1672000	1.9172
2.0505	7.07	1680000	1.9190
2.0505	7.11	1688000	1.9181
2.0548	7.14	1696000	1.9160
2.0548	7.17	1704000	1.9168
2.0524	7.21	1712000	1.9155
2.0524	7.24	1720000	1.9161
2.0539	7.28	1728000	1.9189
2.0539	7.31	1736000	1.9169
2.0542	7.34	1744000	1.9177
2.0542	7.38	1752000	1.9140
2.0509	7.41	1760000	1.9152
2.0509	7.44	1768000	1.9160
2.0507	7.48	1776000	1.9156
2.0507	7.51	1784000	1.9139
2.057	7.55	1792000	1.9140
2.057	7.58	1800000	1.9248
2.0515	7.61	1808000	1.9143
2.0515	7.65	1816000	1.9188
2.0503	7.68	1824000	1.9127
2.0503	7.71	1832000	1.9132
2.0534	7.75	1840000	1.9129
2.0534	7.78	1848000	1.9195
2.0553	7.81	1856000	1.9157
2.0553	7.85	1864000	1.9177
2.0496	7.88	1872000	1.9148
2.0496	7.92	1880000	1.9132
2.0537	7.95	1888000	1.9184
2.0537	7.98	1896000	1.9160
2.0505	8.02	1904000	1.9151
2.0505	8.05	1912000	1.9210
2.0536	8.08	1920000	1.9173
2.0536	8.12	1928000	1.9139
2.0493	8.15	1936000	1.9209
2.0493	8.19	1944000	1.9151
2.052	8.22	1952000	1.9174
2.052	8.25	1960000	1.9146
2.0575	8.29	1968000	1.9169
2.0575	8.32	1976000	1.9173
2.0499	8.35	1984000	1.9175
2.0499	8.39	1992000	1.9136
2.0573	8.42	2000000	1.9159
2.0573	8.45	2008000	1.9148
2.0556	8.49	2016000	1.9174
2.0556	8.52	2024000	1.9146
2.0558	8.56	2032000	1.9152
2.0558	8.59	2040000	1.9125
2.0493	8.62	2048000	1.9156
2.0493	8.66	2056000	1.9121
2.0492	8.69	2064000	1.9227
2.0492	8.72	2072000	1.9136
2.0576	8.76	2080000	1.9147
2.0576	8.79	2088000	1.9159
2.0512	8.83	2096000	1.9116
2.0512	8.86	2104000	1.9159
2.05	8.89	2112000	1.9130
2.05	8.93	2120000	1.9152
2.0437	8.96	2128000	1.9176
2.0437	8.99	2136000	1.9193
2.053	9.03	2144000	1.9124
2.053	9.06	2152000	1.9139
2.0496	9.09	2160000	1.9128
2.0496	9.13	2168000	1.9162
2.0495	9.16	2176000	1.9065
2.0495	9.2	2184000	1.9211
2.0468	9.23	2192000	1.9095
2.0468	9.26	2200000	1.9163
2.0507	9.3	2208000	1.9106
2.0507	9.33	2216000	1.9165
2.0526	9.36	2224000	1.9179
2.0526	9.4	2232000	1.9178
2.0537	9.43	2240000	1.9163
2.0537	9.47	2248000	1.9159
2.0502	9.5	2256000	1.9146
2.0502	9.53	2264000	1.9169
2.0492	9.57	2272000	1.9164
2.0492	9.6	2280000	1.9154
2.0505	9.63	2288000	1.9066
2.0505	9.67	2296000	1.9140
2.0516	9.7	2304000	1.9125
2.0516	9.73	2312000	1.9184
2.0559	9.77	2320000	1.9178
2.0559	9.8	2328000	1.9164
2.0528	9.84	2336000	1.9087
2.0528	9.87	2344000	1.9165
2.0559	9.9	2352000	1.9113
2.0559	9.94	2360000	1.9146
2.058	9.97	2368000	1.9156
2.058	10.0	2376000	1.9137
2.053	10.04	2384000	1.9081
2.053	10.07	2392000	1.9148
2.0566	10.11	2400000	1.9142

Framework versions

• Transformers 4.35.0.dev0
• Pytorch 2.0.1+cu117
• Datasets 2.14.5
• Tokenizers 0.14.0