text
Prompting BERT!
Prompting BERT!
new_line
new_line
text
Zero-shot learning ability is the hottest thing about causal LLMs. You don't need to finetune causal LLMs on each specific task. Instead, you can use prompting and get a decent performance on unseen tasks.
Zero-shot learning ability is the hottest thing about causal LLMs. You don't need to finetune causal LLMs on each specific task. Instead, you can use prompting and get a decent performance on unseen tasks.
new_line
new_line
text
Unfortunately, autoencoding LLMs -  like our dear friend BERT 🙋‍♂️- lack this ability and you need a task-specific head for different tasks. But what if you could prompt all the BERTs in the world?!
Unfortunately, autoencoding LLMs -  like our dear friend BERT 🙋‍♂️- lack this ability and you need a task-specific head for different tasks. But what if you could prompt all the BERTs in the world?!
new_line
new_line
text
🥁 Introducing Statement-Tuning 🥁
🥁 Introducing Statement-Tuning 🥁
new_line
new_line
text
Now hold your horses! don't go full-LLama on it yet. Using this finetuning approach, we can get zero-shot performance from encoders by turning a problem into a yes/no problem. Binary classification all the way down! 
Now hold your horses! don't go full-LLama on it yet. Using this finetuning approach, we can get zero-shot performance from encoders by turning a problem into a yes/no problem. Binary classification all the way down! 
new_line
text
For example, a single entailment problem will be decomposed into 3 yes/no questions. 
For example, a single entailment problem will be decomposed into 3 yes/no questions. 
new_line
new_line
text
This is still not super useful. But I like works that try to make a little more space for encoders in the current autoregressive era!
This is still not super useful. But I like works that try to make a little more space for encoders in the current autoregressive era!
new_line
new_line
text
Check the paper if interested: 
Check the paper if interested: 
resource
https://huggingface.co/papers/2404.12897
paper
2404.12897
https://huggingface.co/papers/2404.12897
Enabling Natural Zero-Shot Prompting on Encoder Models via
  Statement-Tuning (2404.12897)
new_line
Prompting BERT!

Zero-shot learning ability is the hottest thing about causal LLMs. You don't need to finetune causal LLMs on each specific task. Instead, you can use prompting and get a decent performance on unseen tasks.

Unfortunately, autoencoding LLMs -  like our dear friend BERT 🙋‍♂️- lack this ability and you need a task-specific head for different tasks. But what if you could prompt all the BERTs in the world?!

🥁 Introducing Statement-Tuning 🥁

Now hold your horses! don't go full-LLama on it yet. Using this finetuning approach, we can get zero-shot performance from encoders by turning a problem into a yes/no problem. Binary classification all the way down! 
For example, a single entailment problem will be decomposed into 3 yes/no questions. 

This is still not super useful. But I like works that try to make a little more space for encoders in the current autoregressive era!

Check the paper if interested: https://huggingface.co/papers/2404.12897

🔥
radames KingNish Unggi MarinaraSpaghetti