
# FORK of [google/flan-ul2](https://huggingface.co/google/flan-ul2)

> This is a fork of google/flan-ul2 20B implementing a custom `handler.py` for deploying the model to inference-endpoints on a 4x NVIDIA T4.

You can deploy the flan-ul2 with a [1-click](https://ui.endpoints.huggingface.co/new?repository=philschmid/flan-ul2-20b-fp16). 

> **Note: Creation of the endpoint can take 2 hours due super long building process, be patient. We are working on improving this!**

![createEndpoint](createEndpoint.png)


# TL;DR

Flan-UL2 is an encoder decoder model based on the `T5` architecture. It uses the same configuration as the [`UL2 model`](https://huggingface.co/google/ul2)  released earlier last year. It was fine tuned using the "Flan" prompt tuning 
and dataset collection.

According ot the original [blog](https://www.yitay.net/blog/flan-ul2-20b) here are the notable improvements:
- The original UL2 model was only trained with receptive field of 512, which made it non-ideal for N-shot prompting where N is large. 
- The Flan-UL2 checkpoint uses a receptive field of 2048 which makes it more usable for few-shot in-context learning.
- The original UL2 model also had mode switch tokens that was rather mandatory to get good performance. However, they were a little cumbersome as this requires often some changes during inference or finetuning. In this update/change, we continue training UL2 20B for an additional 100k steps (with small batch) to forget “mode tokens” before applying Flan instruction tuning. This Flan-UL2 checkpoint does not require mode tokens anymore.

**Important**: For more details, please see sections 5.2.1 and 5.2.2 of the [paper](https://arxiv.org/pdf/2205.05131v1.pdf).

# Contribution

This model was originally contributed by [Yi Tay](https://www.yitay.net/?author=636616684c5e64780328eece), and added to the Hugging Face ecosystem by [Younes Belkada](https://huggingface.co/ybelkada) & [Arthur Zucker](https://huggingface.co/ArthurZ).

# Citation

If you want to cite this work, please consider citing the [blogpost](https://www.yitay.net/blog/flan-ul2-20b) announcing the release of `Flan-UL2`.

{"license": "apache-2.0", "tags": ["endpoints-template"]}
transformers pytorch t5 text2text-generation endpoints-template arxiv:2205.05131 license:apache-2.0 autotrain_compatible endpoints_compatible has_space text-generation-inference region:us
TAGS
#transformers #pytorch #t5 #text2text-generation #endpoints-template #arxiv-2205.05131 #license-apache-2.0 #autotrain_compatible #endpoints_compatible #has_space #text-generation-inference #region-us 


# FORK of google/flan-ul2

> This is a fork of google/flan-ul2 20B implementing a custom 'URL' for deploying the model to inference-endpoints on a 4x NVIDIA T4.

You can deploy the flan-ul2 with a 1-click. 

> Note: Creation of the endpoint can take 2 hours due super long building process, be patient. We are working on improving this!

!createEndpoint


# TL;DR

Flan-UL2 is an encoder decoder model based on the 'T5' architecture. It uses the same configuration as the 'UL2 model'  released earlier last year. It was fine tuned using the "Flan" prompt tuning 
and dataset collection.

According ot the original blog here are the notable improvements:
- The original UL2 model was only trained with receptive field of 512, which made it non-ideal for N-shot prompting where N is large. 
- The Flan-UL2 checkpoint uses a receptive field of 2048 which makes it more usable for few-shot in-context learning.
- The original UL2 model also had mode switch tokens that was rather mandatory to get good performance. However, they were a little cumbersome as this requires often some changes during inference or finetuning. In this update/change, we continue training UL2 20B for an additional 100k steps (with small batch) to forget “mode tokens” before applying Flan instruction tuning. This Flan-UL2 checkpoint does not require mode tokens anymore.

Important: For more details, please see sections 5.2.1 and 5.2.2 of the paper.

# Contribution

This model was originally contributed by Yi Tay, and added to the Hugging Face ecosystem by Younes Belkada & Arthur Zucker.

If you want to cite this work, please consider citing the blogpost announcing the release of 'Flan-UL2'.

# FORK of google/flan-ul2

> This is a fork of google/flan-ul2 20B implementing a custom 'URL' for deploying the model to inference-endpoints on a 4x NVIDIA T4.

You can deploy the flan-ul2 with a 1-click. 

> Note: Creation of the endpoint can take 2 hours due super long building process, be patient. We are working on improving this!

!createEndpoint # TL;DR

Flan-UL2 is an encoder decoder model based on the 'T5' architecture. It uses the same configuration as the 'UL2 model'  released earlier last year. It was fine tuned using the "Flan" prompt tuning 
and dataset collection.

According ot the original blog here are the notable improvements:
- The original UL2 model was only trained with receptive field of 512, which made it non-ideal for N-shot prompting where N is large. 
- The Flan-UL2 checkpoint uses a receptive field of 2048 which makes it more usable for few-shot in-context learning.
- The original UL2 model also had mode switch tokens that was rather mandatory to get good performance. However, they were a little cumbersome as this requires often some changes during inference or finetuning. In this update/change, we continue training UL2 20B for an additional 100k steps (with small batch) to forget “mode tokens” before applying Flan instruction tuning. This Flan-UL2 checkpoint does not require mode tokens anymore.

Important: For more details, please see sections 5.2.1 and 5.2.2 of the paper. # Contribution

This model was originally contributed by Yi Tay, and added to the Hugging Face ecosystem by Younes Belkada & Arthur Zucker.

If you want to cite this work, please consider citing the blogpost announcing the release of 'Flan-UL2'.
TAGS
#transformers #pytorch #t5 #text2text-generation #endpoints-template #arxiv-2205.05131 #license-apache-2.0 #autotrain_compatible #endpoints_compatible #has_space #text-generation-inference #region-us 
 # FORK of google/flan-ul2

> This is a fork of google/flan-ul2 20B implementing a custom 'URL' for deploying the model to inference-endpoints on a 4x NVIDIA T4.

You can deploy the flan-ul2 with a 1-click. 

> Note: Creation of the endpoint can take 2 hours due super long building process, be patient. We are working on improving this!

!createEndpoint # TL;DR

Flan-UL2 is an encoder decoder model based on the 'T5' architecture. It uses the same configuration as the 'UL2 model'  released earlier last year. It was fine tuned using the "Flan" prompt tuning 
and dataset collection.

According ot the original blog here are the notable improvements:
- The original UL2 model was only trained with receptive field of 512, which made it non-ideal for N-shot prompting where N is large. 
- The Flan-UL2 checkpoint uses a receptive field of 2048 which makes it more usable for few-shot in-context learning.
- The original UL2 model also had mode switch tokens that was rather mandatory to get good performance. However, they were a little cumbersome as this requires often some changes during inference or finetuning. In this update/change, we continue training UL2 20B for an additional 100k steps (with small batch) to forget “mode tokens” before applying Flan instruction tuning. This Flan-UL2 checkpoint does not require mode tokens anymore.

Important: For more details, please see sections 5.2.1 and 5.2.2 of the paper. # Contribution

This model was originally contributed by Yi Tay, and added to the Hugging Face ecosystem by Younes Belkada & Arthur Zucker.

If you want to cite this work, please consider citing the blogpost announcing the release of 'Flan-UL2'.
TAGS
#transformers #pytorch #t5 #text2text-generation #endpoints-template #arxiv-2205.05131 #license-apache-2.0 #autotrain_compatible #endpoints_compatible #has_space #text-generation-inference #region-us 
# FORK of google/flan-ul2

> This is a fork of google/flan-ul2 20B implementing a custom 'URL' for deploying the model to inference-endpoints on a 4x NVIDIA T4.

You can deploy the flan-ul2 with a 1-click. 

> Note: Creation of the endpoint can take 2 hours due super long building process, be patient. We are working on improving this!

!createEndpoint# TL;DR

Flan-UL2 is an encoder decoder model based on the 'T5' architecture. It uses the same configuration as the 'UL2 model'  released earlier last year. It was fine tuned using the "Flan" prompt tuning 
and dataset collection.

According ot the original blog here are the notable improvements:
- The original UL2 model was only trained with receptive field of 512, which made it non-ideal for N-shot prompting where N is large. 
- The Flan-UL2 checkpoint uses a receptive field of 2048 which makes it more usable for few-shot in-context learning.
- The original UL2 model also had mode switch tokens that was rather mandatory to get good performance. However, they were a little cumbersome as this requires often some changes during inference or finetuning. In this update/change, we continue training UL2 20B for an additional 100k steps (with small batch) to forget “mode tokens” before applying Flan instruction tuning. This Flan-UL2 checkpoint does not require mode tokens anymore.

Important: For more details, please see sections 5.2.1 and 5.2.2 of the paper.# Contribution

This model was originally contributed by Yi Tay, and added to the Hugging Face ecosystem by Younes Belkada & Arthur Zucker.

If you want to cite this work, please consider citing the blogpost announcing the release of 'Flan-UL2'.