classification_phobert-v2

This model is a fine-tuned version of vinai/phobert-base-v2 on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.2749
• F1: 0.9467

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 3e-05
• train_batch_size: 128
• eval_batch_size: 128
• seed: 42
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• num_epochs: 40.0

Training results

Training Loss	Epoch	Step	Validation Loss	F1
No log	1.0	194	0.2289	0.9347
No log	2.0	388	0.1812	0.9400
0.2405	3.0	582	0.1774	0.9447
0.2405	4.0	776	0.1997	0.9433
0.2405	5.0	970	0.2236	0.9428
0.1112	6.0	1164	0.2448	0.9380
0.1112	7.0	1358	0.2250	0.9442
0.0717	8.0	1552	0.2410	0.9414
0.0717	9.0	1746	0.2488	0.9414
0.0717	10.0	1940	0.2667	0.9447
0.0525	11.0	2134	0.2683	0.9456
0.0525	12.0	2328	0.3145	0.9414
0.0402	13.0	2522	0.2749	0.9467
0.0402	14.0	2716	0.3030	0.9430
0.0402	15.0	2910	0.3059	0.9458
0.0285	16.0	3104	0.3260	0.9425
0.0285	17.0	3298	0.3208	0.9464
0.0285	18.0	3492	0.3752	0.9394
0.0225	19.0	3686	0.3408	0.9450
0.0225	20.0	3880	0.4128	0.9366
0.0166	21.0	4074	0.3799	0.9408
0.0166	22.0	4268	0.3940	0.9389
0.0166	23.0	4462	0.3740	0.9450
0.0153	24.0	4656	0.3810	0.9400
0.0153	25.0	4850	0.4278	0.9403
0.0122	26.0	5044	0.3878	0.9436
0.0122	27.0	5238	0.3903	0.9433
0.0122	28.0	5432	0.3904	0.9442
0.0114	29.0	5626	0.4205	0.9428
0.0114	30.0	5820	0.3969	0.9433
0.0096	31.0	6014	0.3967	0.9439
0.0096	32.0	6208	0.4009	0.9442
0.0096	33.0	6402	0.4054	0.9439
0.0082	34.0	6596	0.4115	0.9422
0.0082	35.0	6790	0.4228	0.9430
0.0082	36.0	6984	0.4165	0.9442
0.0083	37.0	7178	0.4226	0.9436
0.0083	38.0	7372	0.4262	0.9430
0.0071	39.0	7566	0.4231	0.9436
0.0071	40.0	7760	0.4251	0.9439

Framework versions

• Transformers 4.48.0.dev0
• Pytorch 2.1.0+cu121
• Datasets 3.1.0
• Tokenizers 0.21.0