from transformers import T5ForConditionalGeneration
model = T5ForConditionalGeneration.from_pretrained("google/t5-v1_1-base")

T5 Version 1.1 includes the following improvements compared to the original T5 model:

GEGLU activation in the feed-forward hidden layer, rather than ReLU. See this paper.

Dropout was turned off in pre-training (quality win). Dropout should be re-enabled during fine-tuning.

Pre-trained on C4 only without mixing in the downstream tasks.