File size: 3,606 Bytes
fa04d66 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 |
---
license: apache-2.0
base_model: google/mt5-large
tags:
- generated_from_trainer
metrics:
- rouge
model-index:
- name: MT5-large_NO-idun-20epoch
results: []
---
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->
# MT5-large_NO-idun-20epoch
This model is a fine-tuned version of [google/mt5-large](https://huggingface.co/google/mt5-large) on an unknown dataset.
It achieves the following results on the evaluation set:
- Loss: 1.6704
- Rouge1: 41.2841
- Rouge2: 17.1062
- Rougel: 27.4493
- Rougelsum: 37.2798
- Gen Len: 113.9043
## Model description
More information needed
## Intended uses & limitations
More information needed
## Training and evaluation data
More information needed
## Training procedure
### Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 5e-05
- train_batch_size: 2
- eval_batch_size: 2
- seed: 42
- gradient_accumulation_steps: 4
- total_train_batch_size: 8
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 20
### Training results
| Training Loss | Epoch | Step | Validation Loss | Rouge1 | Rouge2 | Rougel | Rougelsum | Gen Len |
|:-------------:|:-----:|:----:|:---------------:|:-------:|:-------:|:-------:|:---------:|:--------:|
| No log | 0.98 | 46 | 6.3605 | 26.7151 | 6.6971 | 17.105 | 23.7109 | 127.0 |
| No log | 1.99 | 93 | 4.7689 | 32.3429 | 12.2149 | 20.2642 | 28.985 | 127.0 |
| No log | 2.99 | 140 | 1.8494 | 37.8255 | 14.0518 | 22.4306 | 33.4714 | 124.9255 |
| No log | 4.0 | 187 | 1.7294 | 39.5672 | 16.4066 | 24.4606 | 35.4055 | 121.1702 |
| No log | 4.98 | 233 | 1.6796 | 39.5901 | 16.6044 | 25.6316 | 35.5093 | 120.5532 |
| No log | 5.99 | 280 | 1.6557 | 39.8141 | 15.6699 | 24.8691 | 36.0578 | 123.5745 |
| No log | 6.99 | 327 | 1.6525 | 40.0304 | 16.6229 | 25.7054 | 36.3012 | 121.0638 |
| No log | 8.0 | 374 | 1.6484 | 40.5564 | 16.0763 | 26.0131 | 36.1736 | 119.8936 |
| No log | 8.98 | 420 | 1.6499 | 39.9522 | 16.6648 | 26.419 | 35.9155 | 118.9468 |
| No log | 9.99 | 467 | 1.6494 | 41.0085 | 17.1259 | 27.041 | 36.9109 | 115.8085 |
| 3.1043 | 10.99 | 514 | 1.6485 | 41.5339 | 17.5085 | 27.6923 | 37.2051 | 115.8936 |
| 3.1043 | 12.0 | 561 | 1.6488 | 40.3393 | 16.453 | 26.8152 | 36.3384 | 113.4787 |
| 3.1043 | 12.98 | 607 | 1.6485 | 42.0494 | 17.8355 | 27.9197 | 37.9283 | 115.8617 |
| 3.1043 | 13.99 | 654 | 1.6533 | 40.7634 | 16.8655 | 26.8984 | 36.5803 | 114.6809 |
| 3.1043 | 14.99 | 701 | 1.6570 | 41.6789 | 17.5072 | 27.7933 | 37.4503 | 114.1596 |
| 3.1043 | 16.0 | 748 | 1.6594 | 41.5489 | 17.2787 | 27.7975 | 37.2948 | 113.7447 |
| 3.1043 | 16.98 | 794 | 1.6643 | 41.3929 | 17.0913 | 27.4552 | 37.2221 | 113.3936 |
| 3.1043 | 17.99 | 841 | 1.6658 | 41.4336 | 16.9364 | 27.4426 | 37.1709 | 113.1915 |
| 3.1043 | 18.99 | 888 | 1.6699 | 41.5935 | 17.1928 | 27.2885 | 37.2653 | 113.6170 |
| 3.1043 | 19.68 | 920 | 1.6704 | 41.2841 | 17.1062 | 27.4493 | 37.2798 | 113.9043 |
### Framework versions
- Transformers 4.32.1
- Pytorch 2.3.0+cu121
- Datasets 2.12.0
- Tokenizers 0.13.2
|