xlnet_classification

This model is a fine-tuned version of 5CD-AI/Vietnamese-Sentiment-visobert on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.4554
• F1: 0.9481

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 3e-05
• train_batch_size: 128
• eval_batch_size: 128
• seed: 42
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• num_epochs: 40.0

Training results

Training Loss	Epoch	Step	Validation Loss	F1
No log	1.0	194	0.1743	0.9461
No log	2.0	388	0.1837	0.9464
0.1681	3.0	582	0.1892	0.9444
0.1681	4.0	776	0.2257	0.9450
0.1681	5.0	970	0.2677	0.9422
0.0663	6.0	1164	0.2917	0.9369
0.0663	7.0	1358	0.3442	0.9391
0.0377	8.0	1552	0.3365	0.9417
0.0377	9.0	1746	0.3301	0.9411
0.0377	10.0	1940	0.3463	0.9428
0.0259	11.0	2134	0.3746	0.9386
0.0259	12.0	2328	0.4055	0.9436
0.0173	13.0	2522	0.4070	0.9375
0.0173	14.0	2716	0.4143	0.9419
0.0173	15.0	2910	0.4322	0.9411
0.014	16.0	3104	0.4182	0.9436
0.014	17.0	3298	0.4002	0.9442
0.014	18.0	3492	0.4194	0.9453
0.0131	19.0	3686	0.4223	0.9439
0.0131	20.0	3880	0.4358	0.9442
0.0107	21.0	4074	0.4565	0.9411
0.0107	22.0	4268	0.4561	0.9433
0.0107	23.0	4462	0.4366	0.9453
0.0094	24.0	4656	0.4435	0.9425
0.0094	25.0	4850	0.4527	0.9461
0.0091	26.0	5044	0.4759	0.9439
0.0091	27.0	5238	0.4518	0.9464
0.0091	28.0	5432	0.4447	0.9461
0.0085	29.0	5626	0.4533	0.9456
0.0085	30.0	5820	0.4549	0.9458
0.0071	31.0	6014	0.4527	0.9470
0.0071	32.0	6208	0.4533	0.9453
0.0071	33.0	6402	0.4554	0.9481
0.0072	34.0	6596	0.4629	0.9447
0.0072	35.0	6790	0.4726	0.9450
0.0072	36.0	6984	0.4511	0.9461
0.0067	37.0	7178	0.4571	0.9447
0.0067	38.0	7372	0.4676	0.9456
0.0062	39.0	7566	0.4640	0.9461
0.0062	40.0	7760	0.4639	0.9464

Framework versions

• Transformers 4.48.0.dev0
• Pytorch 2.1.0+cu121
• Datasets 3.1.0
• Tokenizers 0.21.0