StefanJevtic63/whisper-large-v2-sr-lora

Whisper - Serbian Model

This model is a fine-tuned version of openai/whisper-large-v2 on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.1118

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0009
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 32
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 400
• training_steps: 16000
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss
0.2185	0.0705	500	0.2191
0.1764	0.1409	1000	0.1864
0.1752	0.2114	1500	0.1786
0.1703	0.2819	2000	0.1690
0.1558	0.3524	2500	0.1629
0.1584	0.4228	3000	0.1586
0.1496	0.4933	3500	0.1547
0.1592	0.5638	4000	0.1520
0.1428	0.6342	4500	0.1496
0.1429	0.7047	5000	0.1459
0.1357	0.7752	5500	0.1443
0.1332	0.8457	6000	0.1406
0.1413	0.9161	6500	0.1398
0.1373	0.9866	7000	0.1369
0.1253	1.0571	7500	0.1345
0.1229	1.1276	8000	0.1330
0.1114	1.1980	8500	0.1316
0.1189	1.2685	9000	0.1292
0.1116	1.3390	9500	0.1266
0.1108	1.4094	10000	0.1258
0.1098	1.4799	10500	0.1243
0.1112	1.5504	11000	0.1223
0.1139	1.6209	11500	0.1212
0.0951	1.6913	12000	0.1200
0.108	1.7618	12500	0.1173
0.104	1.8323	13000	0.1156
0.0963	1.9027	13500	0.1146
0.1004	1.9732	14000	0.1133
0.0831	2.0437	14500	0.1131
0.0779	2.1142	15000	0.1126
0.0805	2.1846	15500	0.1121
0.0849	2.2551	16000	0.1118

Framework versions

• PEFT 0.13.2
• Transformers 4.46.3
• Pytorch 2.5.1
• Datasets 3.0.0
• Tokenizers 0.20.3