transformers tensorboard safetensors t5 text2text-generation generated_from_trainer base_model:tanatapanun/flan-t5-2048-input-1-epochs license:apache-2.0 autotrain_compatible endpoints_compatible text-generation-inference region:us
---
license: apache-2.0
base_model: tanatapanun/flan-t5-2048-input-1-epochs
tags:
- generated_from_trainer
metrics:
- rouge
model-index:
- name: flan-t5-2048-input-30-epochs
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# flan-t5-2048-input-30-epochs

This model is a fine-tuned version of [tanatapanun/flan-t5-2048-input-1-epochs](https://huggingface.co/tanatapanun/flan-t5-2048-input-1-epochs) on the None dataset.
It achieves the following results on the evaluation set:
- Loss: 0.7695
- Rouge1: 0.2731
- Rouge2: 0.1008
- Rougel: 0.2328
- Rougelsum: 0.2319
- Gen Len: 13.82

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 4
- eval_batch_size: 4
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 30

### Training results

| Training Loss | Epoch | Step | Validation Loss | Rouge1 | Rouge2 | Rougel | Rougelsum | Gen Len |
|:-------------:|:-----:|:----:|:---------------:|:------:|:------:|:------:|:---------:|:-------:|
| No log        | 1.0   | 301  | 1.5244          | 0.2018 | 0.0697 | 0.1768 | 0.1758    | 11.98   |
| 2.3092        | 2.0   | 602  | 0.8884          | 0.2155 | 0.0817 | 0.1857 | 0.1837    | 14.36   |
| 2.3092        | 3.0   | 903  | 0.8211          | 0.2112 | 0.0739 | 0.1784 | 0.1772    | 15.2    |
| 0.9906        | 4.0   | 1204 | 0.8055          | 0.2205 | 0.082  | 0.1854 | 0.1844    | 14.71   |
| 0.8522        | 5.0   | 1505 | 0.7995          | 0.2381 | 0.084  | 0.2025 | 0.2026    | 14.59   |
| 0.8522        | 6.0   | 1806 | 0.7937          | 0.2449 | 0.0775 | 0.2046 | 0.2032    | 15.15   |
| 0.8235        | 7.0   | 2107 | 0.7917          | 0.2535 | 0.0882 | 0.2116 | 0.2104    | 14.57   |
| 0.8235        | 8.0   | 2408 | 0.7856          | 0.263  | 0.0848 | 0.2198 | 0.2195    | 14.36   |
| 0.7869        | 9.0   | 2709 | 0.7832          | 0.2759 | 0.097  | 0.2371 | 0.2347    | 13.82   |
| 0.7702        | 10.0  | 3010 | 0.7810          | 0.2622 | 0.0884 | 0.225  | 0.2253    | 14.16   |
| 0.7702        | 11.0  | 3311 | 0.7791          | 0.2807 | 0.0995 | 0.2384 | 0.2376    | 14.11   |
| 0.7721        | 12.0  | 3612 | 0.7786          | 0.2758 | 0.0896 | 0.237  | 0.2362    | 14.17   |
| 0.7721        | 13.0  | 3913 | 0.7772          | 0.2851 | 0.094  | 0.2466 | 0.2456    | 14.05   |
| 0.7534        | 14.0  | 4214 | 0.7747          | 0.2642 | 0.0929 | 0.2245 | 0.2235    | 13.95   |
| 0.7285        | 15.0  | 4515 | 0.7739          | 0.2681 | 0.0993 | 0.2281 | 0.2289    | 13.81   |
| 0.7285        | 16.0  | 4816 | 0.7746          | 0.2714 | 0.0978 | 0.2309 | 0.2302    | 14.02   |
| 0.7334        | 17.0  | 5117 | 0.7727          | 0.2734 | 0.0957 | 0.2317 | 0.2311    | 14.0    |
| 0.7334        | 18.0  | 5418 | 0.7720          | 0.2729 | 0.0996 | 0.2321 | 0.2321    | 13.92   |
| 0.7193        | 19.0  | 5719 | 0.7726          | 0.2693 | 0.0982 | 0.2298 | 0.2291    | 13.88   |
| 0.7147        | 20.0  | 6020 | 0.7727          | 0.2799 | 0.103  | 0.2378 | 0.2368    | 13.91   |
| 0.7147        | 21.0  | 6321 | 0.7715          | 0.2856 | 0.1047 | 0.2465 | 0.2451    | 13.83   |
| 0.7204        | 22.0  | 6622 | 0.7704          | 0.2724 | 0.1016 | 0.2339 | 0.233     | 13.91   |
| 0.7204        | 23.0  | 6923 | 0.7705          | 0.2751 | 0.1017 | 0.2344 | 0.2335    | 13.96   |
| 0.6853        | 24.0  | 7224 | 0.7697          | 0.2746 | 0.1011 | 0.2343 | 0.2333    | 13.73   |
| 0.7116        | 25.0  | 7525 | 0.7703          | 0.2757 | 0.1024 | 0.2357 | 0.2348    | 13.72   |
| 0.7116        | 26.0  | 7826 | 0.7698          | 0.2745 | 0.1008 | 0.2341 | 0.2333    | 13.72   |
| 0.6935        | 27.0  | 8127 | 0.7699          | 0.2731 | 0.1008 | 0.2328 | 0.2319    | 13.82   |
| 0.6935        | 28.0  | 8428 | 0.7698          | 0.2731 | 0.1008 | 0.2328 | 0.2319    | 13.82   |
| 0.7051        | 29.0  | 8729 | 0.7695          | 0.2731 | 0.1008 | 0.2328 | 0.2319    | 13.82   |
| 0.6923        | 30.0  | 9030 | 0.7695          | 0.2731 | 0.1008 | 0.2328 | 0.2319    | 13.82   |


### Framework versions

- Transformers 4.35.2
- Pytorch 2.1.0+cu118
- Datasets 2.15.0
- Tokenizers 0.15.0
