transformers safetensors t5 text2text-generation generated_from_trainer base_model:OmarHaroon01/byt5_pretrained autotrain_compatible endpoints_compatible text-generation-inference region:us
---
base_model: OmarHaroon01/byt5_pretrained
tags:
- generated_from_trainer
metrics:
- rouge
model-index:
- name: byt5_pretrained-samsum
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# byt5_pretrained-samsum

This model is a fine-tuned version of [OmarHaroon01/byt5_pretrained](https://huggingface.co/OmarHaroon01/byt5_pretrained) on an unknown dataset.
It achieves the following results on the evaluation set:
- Loss: 1.4270
- Rouge1: 15.1583
- Rouge2: 1.8812
- Rougel: 12.4182
- Rougelsum: 13.8406
- Gen Len: 93.1673

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 5e-05
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 5

### Training results

| Training Loss | Epoch | Step | Validation Loss | Rouge1  | Rouge2 | Rougel  | Rougelsum | Gen Len |
|:-------------:|:-----:|:----:|:---------------:|:-------:|:------:|:-------:|:---------:|:-------:|
| 1.8451        | 1.0   | 1842 | 1.6530          | 12.5555 | 1.3712 | 10.6513 | 11.568    | 97.8730 |
| 1.7008        | 2.0   | 3684 | 1.5421          | 13.7987 | 1.6109 | 11.5281 | 12.734    | 93.8803 |
| 1.6202        | 3.0   | 5526 | 1.4822          | 14.5607 | 1.5841 | 11.9255 | 13.1058   | 95.8596 |
| 1.568         | 4.0   | 7368 | 1.4416          | 13.9758 | 1.7069 | 11.6157 | 12.8209   | 92.4188 |
| 1.5315        | 5.0   | 9210 | 1.4270          | 15.1583 | 1.8812 | 12.4182 | 13.8406   | 93.1673 |


### Framework versions

- Transformers 4.38.1
- Pytorch 2.1.0+cu121
- Datasets 2.18.0
- Tokenizers 0.15.2
