distilbert-base-cased-airlines-news-multi-label

This model is a fine-tuned version of distilbert/distilbert-base-cased on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.3172
• F1: 0.8338
• Roc Auc: 0.5031

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2.5e-05
• train_batch_size: 24
• eval_batch_size: 24
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 40

Training results

Training Loss	Epoch	Step	Validation Loss	F1	Roc Auc
No log	1.0	72	0.4599	0.8169	0.5
No log	2.0	144	0.3754	0.8169	0.5
No log	3.0	216	0.3603	0.8169	0.5
No log	4.0	288	0.3564	0.8169	0.5
No log	5.0	360	0.3539	0.8169	0.5
No log	6.0	432	0.3517	0.8169	0.5
0.391	7.0	504	0.3497	0.8169	0.5
0.391	8.0	576	0.3475	0.8169	0.5
0.391	9.0	648	0.3456	0.8169	0.5
0.391	10.0	720	0.3435	0.8169	0.5
0.391	11.0	792	0.3413	0.8169	0.5
0.391	12.0	864	0.3394	0.8169	0.5
0.391	13.0	936	0.3373	0.8169	0.5
0.3372	14.0	1008	0.3354	0.8169	0.5
0.3372	15.0	1080	0.3338	0.8169	0.5
0.3372	16.0	1152	0.3322	0.8169	0.5
0.3372	17.0	1224	0.3307	0.8169	0.5
0.3372	18.0	1296	0.3288	0.8169	0.5
0.3372	19.0	1368	0.3275	0.8169	0.5
0.3372	20.0	1440	0.3261	0.8169	0.5
0.3245	21.0	1512	0.3249	0.8169	0.5
0.3245	22.0	1584	0.3235	0.8169	0.5
0.3245	23.0	1656	0.3225	0.8169	0.5
0.3245	24.0	1728	0.3214	0.8169	0.5
0.3245	25.0	1800	0.3204	0.8169	0.5
0.3245	26.0	1872	0.3195	0.8169	0.5
0.3245	27.0	1944	0.3187	0.8169	0.5
0.3152	28.0	2016	0.3179	0.8169	0.5
0.3152	29.0	2088	0.3172	0.8338	0.5031
0.3152	30.0	2160	0.3165	0.8338	0.5031
0.3152	31.0	2232	0.3161	0.8338	0.5031
0.3152	32.0	2304	0.3155	0.8338	0.5031
0.3152	33.0	2376	0.3151	0.8338	0.5031
0.3152	34.0	2448	0.3146	0.8338	0.5031
0.3102	35.0	2520	0.3144	0.8338	0.5031
0.3102	36.0	2592	0.3142	0.8338	0.5031
0.3102	37.0	2664	0.3139	0.8338	0.5031
0.3102	38.0	2736	0.3138	0.8338	0.5031
0.3102	39.0	2808	0.3137	0.8338	0.5031
0.3102	40.0	2880	0.3137	0.8338	0.5031

Framework versions

• Transformers 4.41.1
• Pytorch 2.3.0+cu121
• Datasets 2.19.1
• Tokenizers 0.19.1