transformers tf safetensors t5 text2text-generation generated_from_keras_callback autotrain_compatible endpoints_compatible text-generation-inference region:us
---
tags:
- generated_from_keras_callback
model-index:
- name: t5-base-medium-title-generation
  results: []
---

<!-- This model card has been generated automatically according to the information Keras had access to. You should
probably proofread and complete it, then remove this comment. -->

# t5-base-medium-title-generation

This model was trained from scratch on Medium articles.


## Training and evaluation data

### Dataset
Dataset used was Medium articles: https://www.kaggle.com/datasets/fabiochiusano/medium-articles.

The following are the columns in the dataset:

- title [string]: The title of the article.
- text [string]: The text content of the article.
- url [string]: The URL associated to the article.
- authors [list of strings]: The article authors.
- timestamp [string]: The publication datetime of the article.
- tags [list of strings]: List of tags associated to the article.

The following are the counts of records in the dataset:

- Total dataset size: 192368
- Divided into training (100K), validation (1K), test dataset (1K)
- Preprocessing:
prefix = "summarize: "
max_input_length = 512
max_target_length = 64
Keeping articles with text length at least 500 & title length at least 20
- After preprocessing: training (85639), validation (833), test dataset (850)

### Model description

The model used was T5-base.

### Training procedure

Full finetuning

from transformers import Seq2SeqTrainingArguments
from transformers import Seq2SeqTrainer

### Training hyperparameters

The following hyperparameters were used during training:

minibatch size: 8
learning_rate: 4e-5,
weight_decay: 0.01
num_train_epochs: 1

optimizer: None
training_precision: float32

### Training results

![image/png](https://cdn-uploads.huggingface.co/production/uploads/6432fecf6c6ecd587980768b/J8kpBd-foznlqoqZ-3J1r.png)


### Framework versions

- Transformers 4.38.1
- TensorFlow 2.15.0
- Datasets 2.18.0
- Tokenizers 0.15.2

## Intended uses & limitations

Use at your own discretion