opt-350m_IA3_lr5e-05_bs4_epoch20_wd0.01

This model is a fine-tuned version of facebook/opt-350m on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 3.6235

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 500
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss
4.0247	1.0	157	3.8361
4.0183	2.0	314	3.8183
4.0005	3.0	471	3.7924
3.9764	4.0	628	3.7634
3.8952	5.0	785	3.7394
3.9008	6.0	942	3.7191
3.9332	7.0	1099	3.7025
3.8841	8.0	1256	3.6877
3.8515	9.0	1413	3.6754
3.8297	10.0	1570	3.6650
3.785	11.0	1727	3.6560
3.832	12.0	1884	3.6485
3.7683	13.0	2041	3.6422
3.8607	14.0	2198	3.6369
3.7712	15.0	2355	3.6327
3.7966	16.0	2512	3.6293
3.7841	17.0	2669	3.6266
3.8344	18.0	2826	3.6248
3.7461	19.0	2983	3.6238
3.8097	20.0	3140	3.6235

Framework versions

• Transformers 4.35.2
• Pytorch 2.1.1+cu118
• Datasets 2.15.0
• Tokenizers 0.15.0