2020-Q2-90p-filtered

This model is a fine-tuned version of cardiffnlp/twitter-roberta-base-2019-90m on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 3.5439

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 4.1e-07
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.98) and epsilon=1e-08
• lr_scheduler_type: linear
• training_steps: 2400000

Training results

Training Loss	Epoch	Step	Validation Loss
No log	0.17	8000	4.0640
4.2654	0.34	16000	3.9414
4.2654	0.51	24000	3.8956
4.0459	0.67	32000	3.8527
4.0459	0.84	40000	3.8232
3.9781	1.01	48000	3.7806
3.9781	1.18	56000	3.7861
3.9323	1.35	64000	3.7930
3.9323	1.52	72000	3.7814
3.9224	1.68	80000	3.7815
3.9224	1.85	88000	3.7403
3.8924	2.02	96000	3.7468
3.8924	2.19	104000	3.7400
3.879	2.36	112000	3.7283
3.879	2.53	120000	3.7381
3.8806	2.69	128000	3.7073
3.8806	2.86	136000	3.7083
3.8659	3.03	144000	3.6992
3.8659	3.2	152000	3.6956
3.8634	3.37	160000	3.6745
3.8634	3.54	168000	3.7017
3.8632	3.71	176000	3.6960
3.8632	3.87	184000	3.7202
3.8416	4.04	192000	3.7109
3.8416	4.21	200000	3.6942
3.8368	4.38	208000	3.6944
3.8368	4.55	216000	3.6751
3.8359	4.72	224000	3.6815
3.8359	4.88	232000	3.6915
3.8411	5.05	240000	3.6796
3.8411	5.22	248000	3.6847
3.8359	5.39	256000	3.6988
3.8359	5.56	264000	3.6799
3.8268	5.73	272000	3.6810
3.8268	5.89	280000	3.6639
3.8172	6.06	288000	3.6663
3.8172	6.23	296000	3.6838
3.8263	6.4	304000	3.6756
3.8263	6.57	312000	3.6507
3.8215	6.74	320000	3.6409
3.8215	6.91	328000	3.6790
3.8189	7.07	336000	3.6679
3.8189	7.24	344000	3.6443
3.8155	7.41	352000	3.6588
3.8155	7.58	360000	3.6448
3.8075	7.75	368000	3.6520
3.8075	7.92	376000	3.6541
3.8064	8.08	384000	3.6569
3.8064	8.25	392000	3.6586
3.8092	8.42	400000	3.6701
3.8092	8.59	408000	3.6544
3.8032	8.76	416000	3.6668
3.8032	8.93	424000	3.6631
3.8062	9.09	432000	3.6481
3.8062	9.26	440000	3.6392
3.7987	9.43	448000	3.6482
3.7987	9.6	456000	3.6357
3.7954	9.77	464000	3.6333
3.7954	9.94	472000	3.6653
3.7938	10.11	480000	3.6267
3.7938	10.27	488000	3.6490
3.7901	10.44	496000	3.6417
3.7901	10.61	504000	3.6263
3.7935	10.78	512000	3.6523
3.7935	10.95	520000	3.6444
3.7951	11.12	528000	3.6226
3.7951	11.28	536000	3.6347
3.7861	11.45	544000	3.6372
3.7861	11.62	552000	3.6163
3.7846	11.79	560000	3.6299
3.7846	11.96	568000	3.6330
3.7778	12.13	576000	3.6371
3.7778	12.29	584000	3.6343
3.777	12.46	592000	3.6242
3.777	12.63	600000	3.6119
3.778	12.8	608000	3.6167
3.778	12.97	616000	3.6191
3.7795	13.14	624000	3.6225
3.7795	13.3	632000	3.6056
3.7766	13.47	640000	3.6135
3.7766	13.64	648000	3.6169
3.7729	13.81	656000	3.6035
3.7729	13.98	664000	3.6109
3.7846	14.15	672000	3.6180
3.7846	14.32	680000	3.6171
3.7726	14.48	688000	3.6182
3.7726	14.65	696000	3.6086
3.7717	14.82	704000	3.5852
3.7717	14.99	712000	3.5883
3.7713	15.16	720000	3.6056
3.7713	15.33	728000	3.6004
3.7745	15.49	736000	3.6059
3.7745	15.66	744000	3.6156
3.7557	15.83	752000	3.6029
3.7557	16.0	760000	3.6099
3.7628	16.17	768000	3.6016
3.7628	16.34	776000	3.6008
3.7717	16.5	784000	3.5972
3.7717	16.67	792000	3.5838
3.7616	16.84	800000	3.5868
3.7616	17.01	808000	3.5834
3.7608	17.18	816000	3.6066
3.7608	17.35	824000	3.5911
3.7625	17.52	832000	3.5997
3.7625	17.68	840000	3.5855
3.7634	17.85	848000	3.5861
3.7634	18.02	856000	3.6021
3.75	18.19	864000	3.5966
3.75	18.36	872000	3.5761
3.7492	18.53	880000	3.5757
3.7492	18.69	888000	3.6123
3.7522	18.86	896000	3.5841
3.7522	19.03	904000	3.5831
3.7482	19.2	912000	3.5860
3.7482	19.37	920000	3.5804
3.75	19.54	928000	3.5730
3.75	19.7	936000	3.5955
3.755	19.87	944000	3.5868
3.755	20.04	952000	3.5992
3.7549	20.21	960000	3.5657
3.7549	20.38	968000	3.5780
3.743	20.55	976000	3.5828
3.743	20.72	984000	3.5676
3.75	20.88	992000	3.5724
3.75	21.05	1000000	3.5850
3.7483	21.22	1008000	3.5873
3.7483	21.39	1016000	3.5799
3.7523	21.56	1024000	3.5974
3.7523	21.73	1032000	3.5790
3.7458	21.89	1040000	3.5884
3.7458	22.06	1048000	3.5904
3.7498	22.23	1056000	3.5851
3.7498	22.4	1064000	3.5776
3.7496	22.57	1072000	3.5685
3.7496	22.74	1080000	3.5731
3.7395	22.9	1088000	3.5858
3.7395	23.07	1096000	3.5931
3.7466	23.24	1104000	3.5614
3.7466	23.41	1112000	3.5456
3.7503	23.58	1120000	3.5895
3.7503	23.75	1128000	3.5608
3.7484	23.92	1136000	3.5696
3.7484	24.08	1144000	3.5653
3.7435	24.25	1152000	3.5721
3.7435	24.42	1160000	3.5510
3.7348	24.59	1168000	3.5631
3.7348	24.76	1176000	3.5727
3.7341	24.93	1184000	3.5835
3.7341	25.09	1192000	3.5766
3.7435	25.26	1200000	3.5606
3.7435	25.43	1208000	3.5497
3.732	25.6	1216000	3.5433
3.732	25.77	1224000	3.5420
3.7343	25.94	1232000	3.5987
3.7343	26.1	1240000	3.5956
3.7336	26.27	1248000	3.5673
3.7336	26.44	1256000	3.5643
3.7444	26.61	1264000	3.5848
3.7444	26.78	1272000	3.5693
3.7395	26.95	1280000	3.5745
3.7395	27.12	1288000	3.5758
3.7389	27.28	1296000	3.5685
3.7389	27.45	1304000	3.5712
3.7416	27.62	1312000	3.5693
3.7416	27.79	1320000	3.5740
3.7305	27.96	1328000	3.5803
3.7305	28.13	1336000	3.5682
3.7268	28.29	1344000	3.5928
3.7268	28.46	1352000	3.5608
3.7363	28.63	1360000	3.5587
3.7363	28.8	1368000	3.5603
3.7325	28.97	1376000	3.5711
3.7325	29.14	1384000	3.5828
3.7337	29.3	1392000	3.5790
3.7337	29.47	1400000	3.5795
3.7367	29.64	1408000	3.5528
3.7367	29.81	1416000	3.5766
3.7313	29.98	1424000	3.5610
3.7313	30.15	1432000	3.5834
3.7277	30.32	1440000	3.5546
3.7277	30.48	1448000	3.5534
3.7296	30.65	1456000	3.5646
3.7296	30.82	1464000	3.5436
3.7411	30.99	1472000	3.5778
3.7411	31.16	1480000	3.5541
3.7233	31.33	1488000	3.5720
3.7233	31.49	1496000	3.5567
3.7291	31.66	1504000	3.5477
3.7291	31.83	1512000	3.5557
3.7265	32.0	1520000	3.5643
3.7265	32.17	1528000	3.5739
3.7352	32.34	1536000	3.5628
3.7352	32.5	1544000	3.5542
3.7353	32.67	1552000	3.5496
3.7353	32.84	1560000	3.5737
3.7243	33.01	1568000	3.5788
3.7243	33.18	1576000	3.5631
3.7192	33.35	1584000	3.5438
3.7192	33.52	1592000	3.5554
3.7266	33.68	1600000	3.5748
3.7266	33.85	1608000	3.5620
3.73	34.02	1616000	3.5464
3.73	34.19	1624000	3.5670
3.7264	34.36	1632000	3.5626
3.7264	34.53	1640000	3.5640
3.7317	34.69	1648000	3.5650
3.7317	34.86	1656000	3.5458
3.7332	35.03	1664000	3.5567
3.7332	35.2	1672000	3.5610
3.7248	35.37	1680000	3.5650
3.7248	35.54	1688000	3.5580
3.7232	35.7	1696000	3.5829
3.7232	35.87	1704000	3.5532
3.729	36.04	1712000	3.5723
3.729	36.21	1720000	3.5454
3.7273	36.38	1728000	3.5623
3.7273	36.55	1736000	3.5462
3.7261	36.72	1744000	3.5743
3.7261	36.88	1752000	3.5638
3.7208	37.05	1760000	3.5519
3.7208	37.22	1768000	3.5584
3.7183	37.39	1776000	3.5308
3.7183	37.56	1784000	3.5549
3.7193	37.73	1792000	3.5409
3.7193	37.89	1800000	3.5396
3.7271	38.06	1808000	3.5536
3.7271	38.23	1816000	3.5452
3.7284	38.4	1824000	3.5582
3.7284	38.57	1832000	3.5668
3.714	38.74	1840000	3.5673
3.714	38.9	1848000	3.5477
3.7105	39.07	1856000	3.5662
3.7105	39.24	1864000	3.5498
3.7189	39.41	1872000	3.5493
3.7189	39.58	1880000	3.5676
3.7203	39.75	1888000	3.5640
3.7203	39.91	1896000	3.5747
3.7271	40.08	1904000	3.5592
3.7271	40.25	1912000	3.5515
3.7237	40.42	1920000	3.5704
3.7237	40.59	1928000	3.5642
3.723	40.76	1936000	3.5300
3.723	40.93	1944000	3.5482
3.7224	41.09	1952000	3.5586
3.7224	41.26	1960000	3.5463
3.715	41.43	1968000	3.5323
3.715	41.6	1976000	3.5426
3.7209	41.77	1984000	3.5513
3.7209	41.94	1992000	3.5614
3.7183	42.1	2000000	3.5678
3.7183	42.27	2008000	3.5304
3.7161	42.44	2016000	3.5631
3.7161	42.61	2024000	3.5589
3.7215	42.78	2032000	3.5639
3.7215	42.95	2040000	3.5376
3.7205	43.11	2048000	3.5478
3.7205	43.28	2056000	3.5511
3.7178	43.45	2064000	3.5285
3.7178	43.62	2072000	3.5428
3.7232	43.79	2080000	3.5347
3.7232	43.96	2088000	3.5501
3.7167	44.13	2096000	3.5422
3.7167	44.29	2104000	3.5487
3.7253	44.46	2112000	3.5540
3.7253	44.63	2120000	3.5432
3.7139	44.8	2128000	3.5502
3.7139	44.97	2136000	3.5450
3.7194	45.14	2144000	3.5564
3.7194	45.3	2152000	3.5441
3.7167	45.47	2160000	3.5549
3.7167	45.64	2168000	3.5429
3.7202	45.81	2176000	3.5613
3.7202	45.98	2184000	3.5469
3.7193	46.15	2192000	3.5467
3.7193	46.31	2200000	3.5493
3.717	46.48	2208000	3.5652
3.717	46.65	2216000	3.5669
3.7164	46.82	2224000	3.5755
3.7164	46.99	2232000	3.5580
3.715	47.16	2240000	3.5403
3.715	47.33	2248000	3.5521
3.7091	47.49	2256000	3.5604
3.7091	47.66	2264000	3.5401
3.7199	47.83	2272000	3.5408
3.7199	48.0	2280000	3.5509
3.7238	48.17	2288000	3.5348
3.7238	48.34	2296000	3.5530
3.7193	48.5	2304000	3.5447
3.7193	48.67	2312000	3.5453
3.7195	48.84	2320000	3.5487
3.7195	49.01	2328000	3.5357
3.7187	49.18	2336000	3.5404
3.7187	49.35	2344000	3.5247
3.7157	49.51	2352000	3.5557
3.7157	49.68	2360000	3.5532
3.7144	49.85	2368000	3.5453
3.7144	50.02	2376000	3.5421
3.715	50.19	2384000	3.5183
3.715	50.36	2392000	3.5473
3.7208	50.53	2400000	3.5386

Framework versions

• Transformers 4.35.0.dev0
• Pytorch 2.0.1+cu117
• Datasets 2.14.5
• Tokenizers 0.14.0