transformers tensorboard safetensors t5 text2text-generation generated_from_trainer base_model:google/flan-t5-small license:apache-2.0 autotrain_compatible endpoints_compatible text-generation-inference region:us
---
license: apache-2.0
base_model: google/flan-t5-small
tags:
- generated_from_trainer
metrics:
- rouge
model-index:
- name: fine-tuned-flan-t5-20-epochs-2048-input-256-output
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# fine-tuned-flan-t5-20-epochs-2048-input-256-output

This model is a fine-tuned version of [google/flan-t5-small](https://huggingface.co/google/flan-t5-small) on the None dataset.
It achieves the following results on the evaluation set:
- Loss: 3.1471
- Rouge1: 0.1308
- Rouge2: 0.023
- Rougel: 0.1183
- Rougelsum: 0.1188
- Gen Len: 103.96

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 0.0001
- train_batch_size: 4
- eval_batch_size: 4
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 20

### Training results

| Training Loss | Epoch | Step | Validation Loss | Rouge1 | Rouge2 | Rougel | Rougelsum | Gen Len |
|:-------------:|:-----:|:----:|:---------------:|:------:|:------:|:------:|:---------:|:-------:|
| No log        | 1.0   | 301  | 5.5699          | 0.0309 | 0.0079 | 0.0275 | 0.0279    | 167.41  |
| 8.7004        | 2.0   | 602  | 5.0463          | 0.0629 | 0.0101 | 0.0632 | 0.0638    | 135.78  |
| 8.7004        | 3.0   | 903  | 4.0270          | 0.0471 | 0.0049 | 0.0468 | 0.0463    | 205.06  |
| 6.1746        | 4.0   | 1204 | 3.7187          | 0.0739 | 0.0101 | 0.0691 | 0.0702    | 88.45   |
| 5.1998        | 5.0   | 1505 | 3.3997          | 0.0564 | 0.0097 | 0.0511 | 0.0519    | 174.76  |
| 5.1998        | 6.0   | 1806 | 3.1995          | 0.0963 | 0.0195 | 0.0878 | 0.0884    | 108.71  |
| 4.6352        | 7.0   | 2107 | 3.1787          | 0.0978 | 0.0159 | 0.089  | 0.0893    | 143.4   |
| 4.6352        | 8.0   | 2408 | 3.1274          | 0.1123 | 0.0184 | 0.1037 | 0.1035    | 133.42  |
| 4.0979        | 9.0   | 2709 | 2.9934          | 0.0885 | 0.0169 | 0.0818 | 0.0811    | 136.61  |
| 3.7568        | 10.0  | 3010 | 2.9458          | 0.121  | 0.0154 | 0.1134 | 0.1122    | 141.13  |
| 3.7568        | 11.0  | 3311 | 2.9357          | 0.1232 | 0.0186 | 0.1119 | 0.1122    | 136.52  |
| 3.5713        | 12.0  | 3612 | 2.9760          | 0.1127 | 0.0199 | 0.1011 | 0.1009    | 96.31   |
| 3.5713        | 13.0  | 3913 | 2.9262          | 0.0962 | 0.0135 | 0.0854 | 0.0848    | 136.75  |
| 3.2308        | 14.0  | 4214 | 2.9597          | 0.1213 | 0.0248 | 0.1118 | 0.1122    | 125.09  |
| 3.0663        | 15.0  | 4515 | 3.0330          | 0.1054 | 0.019  | 0.0941 | 0.0934    | 130.3   |
| 3.0663        | 16.0  | 4816 | 3.0490          | 0.126  | 0.0203 | 0.1125 | 0.1137    | 123.51  |
| 2.9285        | 17.0  | 5117 | 3.0463          | 0.1215 | 0.0151 | 0.1086 | 0.1087    | 106.23  |
| 2.9285        | 18.0  | 5418 | 3.1519          | 0.1278 | 0.0195 | 0.1142 | 0.1137    | 108.3   |
| 2.6943        | 19.0  | 5719 | 3.1072          | 0.1338 | 0.017  | 0.1204 | 0.1206    | 105.96  |
| 2.7837        | 20.0  | 6020 | 3.1471          | 0.1308 | 0.023  | 0.1183 | 0.1188    | 103.96  |


### Framework versions

- Transformers 4.36.2
- Pytorch 1.12.1+cu113
- Datasets 2.16.1
- Tokenizers 0.15.0
