
## ptune-FLAN-OPT-6.7b

OPT was first introduced in [Open Pre-trained Transformer Language Models](https://arxiv.org/abs/2205.01068) and first released in [metaseq's repository](https://github.com/facebookresearch/metaseq) on May 3rd 2022 by Meta AI.

This model is [facebook/opt-6.7b](https://hf.co/facebook/opt-6.7b) finetuned with prefix tuning (https://arxiv.org/abs/2101.00190) on the FLAN datasets (https://arxiv.org/pdf/2210.11416.pdf).

A 24 token prefix was finetuned over 1.5m new tokens of a FLAN task mixture, with the start of each example cut off if it was too large to fit within a 256 token context.

The model reaches a train ppl of 6.09 and an eval ppl of 5.91.

### Example COT (Chain-of-thought) Prompt:

```
Q: Answer the following yes/no question by reasoning step-by-step. Could a dandelion suffer from hepatitis?
A: Hepatitis only affects organisms with livers. Dandelions don’t have a liver. The answer is no.

Q: Answer the following yes/no question by reasoning step-by-step. Can you write a whole Haiku in a single tweet?
A: A haiku is a japanese three-line poem. That is short enough to fit in 280 characters. The answer is yes.

Q: Answer the following yes/no question by reasoning step-by-step. Can you reach space with a Cessna?
A:
```
``` 
> A Cessna is a small plane that can carry up to 6 people. The answer is no.
```

(Completed with Contrastive Sampling, top_k: 4, penalty_alpha: 0.6)
{"tags": ["flan", "opt", "peft"], "datasets": ["SirNeural/flan_v2"], "metrics": ["perplexity"]}
peft flan opt dataset:SirNeural/flan_v2 arxiv:2205.01068 arxiv:2101.00190 arxiv:2210.11416 region:us
TAGS
#peft #flan #opt #dataset-SirNeural/flan_v2 #arxiv-2205.01068 #arxiv-2101.00190 #arxiv-2210.11416 #region-us 


## ptune-FLAN-OPT-6.7b

OPT was first introduced in Open Pre-trained Transformer Language Models and first released in metaseq's repository on May 3rd 2022 by Meta AI.

This model is facebook/opt-6.7b finetuned with prefix tuning (URL on the FLAN datasets (URL

A 24 token prefix was finetuned over 1.5m new tokens of a FLAN task mixture, with the start of each example cut off if it was too large to fit within a 256 token context.

The model reaches a train ppl of 6.09 and an eval ppl of 5.91.

### Example COT (Chain-of-thought) Prompt:




(Completed with Contrastive Sampling, top_k: 4, penalty_alpha: 0.6)
## ptune-FLAN-OPT-6.7b

OPT was first introduced in Open Pre-trained Transformer Language Models and first released in metaseq's repository on May 3rd 2022 by Meta AI.

This model is facebook/opt-6.7b finetuned with prefix tuning (URL on the FLAN datasets (URL

A 24 token prefix was finetuned over 1.5m new tokens of a FLAN task mixture, with the start of each example cut off if it was too large to fit within a 256 token context.

The model reaches a train ppl of 6.09 and an eval ppl of 5.91. ### Example COT (Chain-of-thought) Prompt:




(Completed with Contrastive Sampling, top_k: 4, penalty_alpha: 0.6)
TAGS
#peft #flan #opt #dataset-SirNeural/flan_v2 #arxiv-2205.01068 #arxiv-2101.00190 #arxiv-2210.11416 #region-us 
 ## ptune-FLAN-OPT-6.7b

OPT was first introduced in Open Pre-trained Transformer Language Models and first released in metaseq's repository on May 3rd 2022 by Meta AI.

This model is facebook/opt-6.7b finetuned with prefix tuning (URL on the FLAN datasets (URL

A 24 token prefix was finetuned over 1.5m new tokens of a FLAN task mixture, with the start of each example cut off if it was too large to fit within a 256 token context.

The model reaches a train ppl of 6.09 and an eval ppl of 5.91. ### Example COT (Chain-of-thought) Prompt:




(Completed with Contrastive Sampling, top_k: 4, penalty_alpha: 0.6)
TAGS
#peft #flan #opt #dataset-SirNeural/flan_v2 #arxiv-2205.01068 #arxiv-2101.00190 #arxiv-2210.11416 #region-us 
## ptune-FLAN-OPT-6.7b

OPT was first introduced in Open Pre-trained Transformer Language Models and first released in metaseq's repository on May 3rd 2022 by Meta AI.

This model is facebook/opt-6.7b finetuned with prefix tuning (URL on the FLAN datasets (URL

A 24 token prefix was finetuned over 1.5m new tokens of a FLAN task mixture, with the start of each example cut off if it was too large to fit within a 256 token context.

The model reaches a train ppl of 6.09 and an eval ppl of 5.91.### Example COT (Chain-of-thought) Prompt:




(Completed with Contrastive Sampling, top_k: 4, penalty_alpha: 0.6)