transformers safetensors t5 text2text-generation generated_from_trainer base_model:t5-small license:apache-2.0 autotrain_compatible endpoints_compatible text-generation-inference region:us
---
license: apache-2.0
base_model: t5-small
tags:
- generated_from_trainer
metrics:
- rouge
model-index:
- name: t5-small-finetuned-laws_articles
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# t5-small-finetuned-laws_articles

This model is a fine-tuned version of [t5-small](https://huggingface.co/t5-small) on the None dataset.
It achieves the following results on the evaluation set:
- Loss: 2.4459
- Rouge1: 28.5123
- Rouge2: 10.7847
- Rougel: 23.1779
- Rougelsum: 23.1054
- Gen Len: 18.7143

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 16
- eval_batch_size: 16
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 50
- mixed_precision_training: Native AMP

### Training results

| Training Loss | Epoch | Step | Validation Loss | Rouge1  | Rouge2  | Rougel  | Rougelsum | Gen Len |
|:-------------:|:-----:|:----:|:---------------:|:-------:|:-------:|:-------:|:---------:|:-------:|
| No log        | 1.0   | 87   | 3.0316          | 27.6181 | 11.4154 | 23.0601 | 23.0895   | 18.6494 |
| No log        | 2.0   | 174  | 2.8530          | 28.2521 | 11.5239 | 23.2405 | 23.2335   | 18.5325 |
| No log        | 3.0   | 261  | 2.7724          | 28.1619 | 10.4465 | 22.4885 | 22.5004   | 18.4545 |
| No log        | 4.0   | 348  | 2.7206          | 27.6965 | 10.4247 | 22.5084 | 22.504    | 18.5714 |
| No log        | 5.0   | 435  | 2.6820          | 28.0781 | 10.4734 | 23.0002 | 22.9096   | 18.5455 |
| 3.0366        | 6.0   | 522  | 2.6535          | 28.2097 | 10.5925 | 23.4952 | 23.474    | 18.4675 |
| 3.0366        | 7.0   | 609  | 2.6330          | 27.9031 | 10.4349 | 23.1427 | 23.1499   | 18.5325 |
| 3.0366        | 8.0   | 696  | 2.6191          | 27.7764 | 10.1563 | 22.7929 | 22.6971   | 18.4286 |
| 3.0366        | 9.0   | 783  | 2.5998          | 27.2375 | 9.7613  | 22.2731 | 22.2206   | 18.2727 |
| 3.0366        | 10.0  | 870  | 2.5866          | 27.2295 | 9.6843  | 22.5222 | 22.3697   | 18.3377 |
| 3.0366        | 11.0  | 957  | 2.5723          | 26.7174 | 9.3882  | 22.2336 | 22.0906   | 18.3377 |
| 2.7071        | 12.0  | 1044 | 2.5651          | 27.4541 | 10.5671 | 22.7291 | 22.6063   | 18.2468 |
| 2.7071        | 13.0  | 1131 | 2.5539          | 27.1483 | 9.9641  | 22.252  | 22.0612   | 18.2857 |
| 2.7071        | 14.0  | 1218 | 2.5451          | 27.4547 | 10.3266 | 22.6335 | 22.4141   | 18.3247 |
| 2.7071        | 15.0  | 1305 | 2.5366          | 27.3057 | 10.0068 | 22.5939 | 22.4321   | 18.3377 |
| 2.7071        | 16.0  | 1392 | 2.5291          | 27.0093 | 9.8964  | 22.1854 | 22.0594   | 18.4416 |
| 2.7071        | 17.0  | 1479 | 2.5210          | 27.4076 | 9.8442  | 22.4911 | 22.3641   | 18.3117 |
| 2.5937        | 18.0  | 1566 | 2.5130          | 27.8067 | 10.1345 | 22.5757 | 22.4506   | 18.3896 |
| 2.5937        | 19.0  | 1653 | 2.5086          | 28.5205 | 10.8396 | 22.902  | 22.7628   | 18.4416 |
| 2.5937        | 20.0  | 1740 | 2.5043          | 28.1191 | 10.3793 | 22.6607 | 22.5025   | 18.5584 |
| 2.5937        | 21.0  | 1827 | 2.4971          | 28.0615 | 10.3535 | 22.7174 | 22.5795   | 18.6104 |
| 2.5937        | 22.0  | 1914 | 2.4935          | 28.0575 | 10.2809 | 22.9132 | 22.7487   | 18.5974 |
| 2.5208        | 23.0  | 2001 | 2.4886          | 27.8841 | 10.3683 | 22.66   | 22.5543   | 18.7532 |
| 2.5208        | 24.0  | 2088 | 2.4851          | 27.5211 | 9.9966  | 22.5687 | 22.4658   | 18.6753 |
| 2.5208        | 25.0  | 2175 | 2.4823          | 27.7727 | 10.2515 | 22.7401 | 22.5633   | 18.5974 |
| 2.5208        | 26.0  | 2262 | 2.4785          | 28.042  | 10.4358 | 22.9115 | 22.7964   | 18.7273 |
| 2.5208        | 27.0  | 2349 | 2.4731          | 27.6472 | 10.2882 | 22.6181 | 22.4979   | 18.6364 |
| 2.5208        | 28.0  | 2436 | 2.4714          | 27.8088 | 10.4361 | 22.885  | 22.7203   | 18.6623 |
| 2.4649        | 29.0  | 2523 | 2.4694          | 27.73   | 10.5222 | 22.7567 | 22.6331   | 18.7013 |
| 2.4649        | 30.0  | 2610 | 2.4660          | 27.6279 | 10.377  | 22.4668 | 22.4061   | 18.7013 |
| 2.4649        | 31.0  | 2697 | 2.4649          | 27.3504 | 10.1574 | 22.224  | 22.1656   | 18.7013 |
| 2.4649        | 32.0  | 2784 | 2.4639          | 27.3448 | 9.9756  | 22.1962 | 22.1349   | 18.7013 |
| 2.4649        | 33.0  | 2871 | 2.4609          | 27.3083 | 10.0656 | 22.2803 | 22.2228   | 18.7143 |
| 2.4649        | 34.0  | 2958 | 2.4593          | 27.51   | 10.0941 | 22.3699 | 22.3382   | 18.6623 |
| 2.4181        | 35.0  | 3045 | 2.4562          | 27.5434 | 10.1665 | 22.5002 | 22.4415   | 18.7143 |
| 2.4181        | 36.0  | 3132 | 2.4545          | 27.5664 | 10.0953 | 22.5101 | 22.4741   | 18.6364 |
| 2.4181        | 37.0  | 3219 | 2.4530          | 27.5874 | 10.333  | 22.5099 | 22.4576   | 18.7013 |
| 2.4181        | 38.0  | 3306 | 2.4522          | 27.3705 | 10.1022 | 22.4695 | 22.3849   | 18.6623 |
| 2.4181        | 39.0  | 3393 | 2.4512          | 27.8864 | 10.2695 | 22.834  | 22.759    | 18.6753 |
| 2.4181        | 40.0  | 3480 | 2.4504          | 27.6797 | 10.2472 | 22.5788 | 22.5353   | 18.7013 |
| 2.4038        | 41.0  | 3567 | 2.4495          | 27.6797 | 10.2472 | 22.5788 | 22.5353   | 18.7013 |
| 2.4038        | 42.0  | 3654 | 2.4492          | 27.1595 | 9.6186  | 22.1003 | 22.0487   | 18.6883 |
| 2.4038        | 43.0  | 3741 | 2.4490          | 27.8341 | 10.2826 | 22.8117 | 22.7244   | 18.7013 |
| 2.4038        | 44.0  | 3828 | 2.4479          | 28.0005 | 10.4067 | 22.8833 | 22.8177   | 18.7013 |
| 2.4038        | 45.0  | 3915 | 2.4475          | 28.0811 | 10.4117 | 22.8975 | 22.8276   | 18.7013 |
| 2.3785        | 46.0  | 4002 | 2.4471          | 28.0811 | 10.4117 | 22.8975 | 22.8276   | 18.7013 |
| 2.3785        | 47.0  | 4089 | 2.4466          | 28.4435 | 10.793  | 23.0757 | 23.0082   | 18.7013 |
| 2.3785        | 48.0  | 4176 | 2.4463          | 28.5123 | 10.7847 | 23.1779 | 23.1054   | 18.7143 |
| 2.3785        | 49.0  | 4263 | 2.4460          | 28.2551 | 10.653  | 22.9815 | 22.8896   | 18.7143 |
| 2.3785        | 50.0  | 4350 | 2.4459          | 28.5123 | 10.7847 | 23.1779 | 23.1054   | 18.7143 |


### Framework versions

- Transformers 4.35.0
- Pytorch 2.1.0+cu121
- Datasets 2.14.7
- Tokenizers 0.14.1
