SChem5Labels-google-t5-v1_1-large-intra_model-dataset-frequency-model_annots_str

This model is a fine-tuned version of google/t5-v1_1-large on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.6279

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 128
• eval_batch_size: 128
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 200

Training results

Training Loss	Epoch	Step	Validation Loss
19.7945	1.0	25	24.1161
18.755	2.0	50	19.9773
18.2706	3.0	75	11.6905
16.8867	4.0	100	10.8195
15.645	5.0	125	10.3424
12.6568	6.0	150	9.3585
10.9942	7.0	175	9.1290
9.2494	8.0	200	8.8652
8.4954	9.0	225	8.5857
8.1042	10.0	250	8.4259
7.8977	11.0	275	8.3043
7.8384	12.0	300	8.1858
7.7411	13.0	325	7.9134
7.3565	14.0	350	7.6255
7.2074	15.0	375	7.3867
7.0111	16.0	400	7.2259
6.9705	17.0	425	7.1460
6.8314	18.0	450	7.0866
6.7505	19.0	475	7.0398
6.6081	20.0	500	6.9983
6.7054	21.0	525	6.9523
6.5904	22.0	550	6.9090
6.4272	23.0	575	6.5798
0.9712	24.0	600	0.7144
0.7214	25.0	625	0.6178
0.6687	26.0	650	0.6174
0.6565	27.0	675	0.6148
0.6602	28.0	700	0.6140
0.6449	29.0	725	0.6121
0.648	30.0	750	0.6133
0.6425	31.0	775	0.6154
0.6505	32.0	800	0.6115
0.661	33.0	825	0.6128
0.6482	34.0	850	0.6108
0.6501	35.0	875	0.6137
0.6436	36.0	900	0.6086
0.6377	37.0	925	0.6107
0.6275	38.0	950	0.6116
0.6254	39.0	975	0.6113
0.6357	40.0	1000	0.6091
0.6443	41.0	1025	0.6095

Framework versions

• Transformers 4.34.0
• Pytorch 2.1.0+cu121
• Datasets 2.14.5
• Tokenizers 0.14.1