metadata

license: apache-2.0
base_model: google/mt5-large
tags:
  - generated_from_keras_callback
model-index:
  - name: pakawadeep/mt5-large-finetuned-ctfl
    results: []

pakawadeep/mt5-large-finetuned-ctfl

This model is a fine-tuned version of google/mt5-large on an unknown dataset. It achieves the following results on the evaluation set:

Train Loss: 1.4894
Validation Loss: 1.1993
Train Rouge1: 8.0858
Train Rouge2: 1.5677
Train Rougel: 7.9208
Train Rougelsum: 8.1271
Train Gen Len: 11.6139
Epoch: 15

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

optimizer: {'name': 'AdamWeightDecay', 'learning_rate': 2e-05, 'decay': 0.0, 'beta_1': 0.9, 'beta_2': 0.999, 'epsilon': 1e-07, 'amsgrad': False, 'weight_decay_rate': 0.01}
training_precision: float32

Training results

Train Loss	Validation Loss	Train Rouge1	Train Rouge2	Train Rougel	Train Rougelsum	Train Gen Len	Epoch
11.3596	5.2319	2.5366	0.5088	2.5148	2.4929	19.0	0
6.1803	3.1508	2.7057	0.5265	2.6846	2.6643	19.0	1
4.6767	2.5774	2.7471	0.5265	2.7054	2.6899	18.3218	2
3.8698	2.8216	2.9763	0.2200	2.8792	2.8987	16.6238	3
4.7045	2.7911	3.2793	0.5501	3.1484	3.2486	14.1881	4
4.0342	2.4191	5.9406	0.6365	5.7206	5.8306	10.6980	5
3.4642	2.1307	5.9406	0.9406	5.7756	5.8463	11.2228	6
3.0690	1.9079	6.0644	0.9901	5.9406	6.0644	11.2228	7
2.6140	1.7092	5.7756	0.8251	5.6518	5.8168	11.4604	8
2.4520	1.6478	5.8581	0.8251	5.6931	5.8581	11.0842	9
2.2701	1.5641	5.9406	0.8251	5.8581	5.8581	10.8465	10
2.0735	1.4839	7.3020	1.0726	7.1370	7.2814	11.0891	11
1.8757	1.3780	7.4670	1.0726	7.3020	7.4257	11.2228	12
1.7313	1.3204	7.3020	1.0726	7.1370	7.2814	11.5842	13
1.5944	1.2466	7.4670	1.0726	7.3020	7.4257	11.6485	14
1.4894	1.1993	8.0858	1.5677	7.9208	8.1271	11.6139	15

Framework versions

Transformers 4.38.2
TensorFlow 2.15.0
Datasets 2.18.0
Tokenizers 0.15.2