transformers tensorboard safetensors t5 text2text-generation generated_from_trainer dataset:xsum base_model:t5-small license:apache-2.0 model-index autotrain_compatible endpoints_compatible text-generation-inference region:us
---
license: apache-2.0
base_model: t5-small
tags:
- generated_from_trainer
datasets:
- xsum
metrics:
- rouge
model-index:
- name: t5-small-finetuned-xsum-updated
  results:
  - task:
      name: Sequence-to-sequence Language Modeling
      type: text2text-generation
    dataset:
      name: xsum
      type: xsum
      config: default
      split: validation
      args: default
    metrics:
    - name: Rouge1
      type: rouge
      value: 33.2945
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# t5-small-finetuned-xsum-updated

This model is a fine-tuned version of [t5-small](https://huggingface.co/t5-small) on the xsum dataset.
It achieves the following results on the evaluation set:
- Loss: 2.0767
- Rouge1: 33.2945
- Rouge2: 12.0165
- Rougel: 26.9804
- Rougelsum: 26.9729
- Gen Len: 18.7853

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 0.0001
- train_batch_size: 16
- eval_batch_size: 16
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 15
- mixed_precision_training: Native AMP

### Training results

| Training Loss | Epoch | Step   | Validation Loss | Rouge1  | Rouge2  | Rougel  | Rougelsum | Gen Len |
|:-------------:|:-----:|:------:|:---------------:|:-------:|:-------:|:-------:|:---------:|:-------:|
| 2.5219        | 1.0   | 12753  | 2.3054          | 30.4745 | 9.435   | 24.263  | 24.2522   | 18.823  |
| 2.4191        | 2.0   | 25506  | 2.2385          | 31.2305 | 10.0552 | 24.9345 | 24.9254   | 18.7562 |
| 2.3564        | 3.0   | 38259  | 2.1961          | 31.8234 | 10.6556 | 25.6109 | 25.6023   | 18.7708 |
| 2.3028        | 4.0   | 51012  | 2.1692          | 32.2053 | 11.0513 | 26.0184 | 26.0056   | 18.772  |
| 2.2737        | 5.0   | 63765  | 2.1452          | 32.3716 | 11.1779 | 26.1423 | 26.1363   | 18.7731 |
| 2.2432        | 6.0   | 76518  | 2.1304          | 32.5413 | 11.2517 | 26.2119 | 26.2098   | 18.8007 |
| 2.2266        | 7.0   | 89271  | 2.1193          | 32.8983 | 11.5683 | 26.5995 | 26.5958   | 18.8108 |
| 2.1863        | 8.0   | 102024 | 2.1058          | 32.9046 | 11.6564 | 26.6466 | 26.6473   | 18.8008 |
| 2.1583        | 9.0   | 114777 | 2.0987          | 32.9622 | 11.7285 | 26.7161 | 26.7116   | 18.7798 |
| 2.1653        | 10.0  | 127530 | 2.0900          | 33.1259 | 11.8525 | 26.8461 | 26.8419   | 18.7999 |
| 2.1403        | 11.0  | 140283 | 2.0880          | 33.0949 | 11.8135 | 26.7863 | 26.7765   | 18.7629 |
| 2.1212        | 12.0  | 153036 | 2.0825          | 33.1671 | 11.8939 | 26.9072 | 26.8982   | 18.7825 |
| 2.1021        | 13.0  | 165789 | 2.0793          | 33.1375 | 11.9119 | 26.8466 | 26.8386   | 18.8076 |
| 2.0877        | 14.0  | 178542 | 2.0774          | 33.2516 | 11.9574 | 26.9391 | 26.9327   | 18.7989 |
| 2.0984        | 15.0  | 191295 | 2.0767          | 33.2945 | 12.0165 | 26.9804 | 26.9729   | 18.7853 |


### Framework versions

- Transformers 4.35.2
- Pytorch 2.1.0+cu121
- Datasets 2.15.0
- Tokenizers 0.15.0
