
[Google's T5-v1.1-base](https://huggingface.co/google/t5-v1_1-base) pre-trained for 24 hours (80k steps / 256 batch size) on a single GPU in [nanoT5](https://github.com/PiotrNawrot/nanoT5) library for efficient pre-training.

For more details about the model refer to the original [paper](https://arxiv.org/pdf/2002.05202.pdf) and original [model weights](https://huggingface.co/google/t5-v1_1-base).

It can be further fine-tuned on SuperNatural-Instructions dataset to achieve comparable performance to the same model pre-trained on 150x more data through "a combination of model and data parallelism [...] on slices of Cloud TPU Pods", each with 1024 TPUs.
{"language": ["en"], "license": "apache-2.0", "tags": ["nanoT5"], "datasets": ["allenai/c4"], "metrics": ["rouge"]}
transformers pytorch t5 text2text-generation nanoT5 en dataset:allenai/c4 arxiv:2002.05202 license:apache-2.0 autotrain_compatible endpoints_compatible text-generation-inference region:us
TAGS
#transformers #pytorch #t5 #text2text-generation #nanoT5 #en #dataset-allenai/c4 #arxiv-2002.05202 #license-apache-2.0 #autotrain_compatible #endpoints_compatible #text-generation-inference #region-us 


Google's T5-v1.1-base pre-trained for 24 hours (80k steps / 256 batch size) on a single GPU in nanoT5 library for efficient pre-training.

For more details about the model refer to the original paper and original model weights.

It can be further fine-tuned on SuperNatural-Instructions dataset to achieve comparable performance to the same model pre-trained on 150x more data through "a combination of model and data parallelism [...] on slices of Cloud TPU Pods", each with 1024 TPUs.
TAGS
#transformers #pytorch #t5 #text2text-generation #nanoT5 #en #dataset-allenai/c4 #arxiv-2002.05202 #license-apache-2.0 #autotrain_compatible #endpoints_compatible #text-generation-inference #region-us 

TAGS
#transformers #pytorch #t5 #text2text-generation #nanoT5 #en #dataset-allenai/c4 #arxiv-2002.05202 #license-apache-2.0 #autotrain_compatible #endpoints_compatible #text-generation-inference #region-us 
