text
"A Closer Look at the Limitations of Instruction Tuning" is a new paper that explores the efficacy and limitations of Instruction Tuning (IT) in Large Language Models (LLMs) for conversational agents. The authors conduct a series of experiments using both LoRA fine-tuning (LFT) and standard full-parameter fine-tuning (SFT) across various LLMs and IT datasets. 
"A Closer Look at the Limitations of Instruction Tuning" is a new paper that explores the efficacy and limitations of Instruction Tuning (IT) in Large Language Models (LLMs) for conversational agents. The authors conduct a series of experiments using both LoRA fine-tuning (LFT) and standard full-parameter fine-tuning (SFT) across various LLMs and IT datasets. 
new_line
new_line
text
The key findings are:
The key findings are:
new_line
text
* LoRA fine-tuning (LFT) preserves the pre-training token distribution while SFT doesn't. This indicates that using LFT, post fine-tuning the model still heavily relies on the pre-training and doesn't acquire new information.
* LoRA fine-tuning (LFT) preserves the pre-training token distribution while SFT doesn't. This indicates that using LFT, post fine-tuning the model still heavily relies on the pre-training and doesn't acquire new information.
new_line
text
* Dataset scaling is ineffective for LFT - experiments show that scaling the dataset size 52x or even 326x doesn't improve the performance.
* Dataset scaling is ineffective for LFT - experiments show that scaling the dataset size 52x or even 326x doesn't improve the performance.
new_line
text
* LoRA fine-tuning mainly enhances response initiation and style without substantial knowledge enhancement.
* LoRA fine-tuning mainly enhances response initiation and style without substantial knowledge enhancement.
new_line
text
* Full-parameter fine-tuning tends to degrade LLM knowledge base and increase hallucination occurrences.
* Full-parameter fine-tuning tends to degrade LLM knowledge base and increase hallucination occurrences.
new_line
text
* Popular other methods and adjustments fail to significantly outperform simple LoRA fine-tuned models in terms of conversational quality and accuracy.
* Popular other methods and adjustments fail to significantly outperform simple LoRA fine-tuned models in terms of conversational quality and accuracy.
new_line
new_line
text
Congrats to the authors 
Congrats to the authors 
mention
@Sreyan88
Sreyan88
text
 and others for their work!
 and others for their work!
new_line
new_line
text
Paper: 
Paper: 
resource
https://huggingface.co/papers/2402.05119
paper
2402.05119
https://huggingface.co/papers/2402.05119
A Closer Look at the Limitations of Instruction Tuning (2402.05119)
"A Closer Look at the Limitations of Instruction Tuning" is a new paper that explores the efficacy and limitations of Instruction Tuning (IT) in Large Language Models (LLMs) for conversational agents. The authors conduct a series of experiments using both LoRA fine-tuning (LFT) and standard full-parameter fine-tuning (SFT) across various LLMs and IT datasets. 

The key findings are:
* LoRA fine-tuning (LFT) preserves the pre-training token distribution while SFT doesn't. This indicates that using LFT, post fine-tuning the model still heavily relies on the pre-training and doesn't acquire new information.
* Dataset scaling is ineffective for LFT - experiments show that scaling the dataset size 52x or even 326x doesn't improve the performance.
* LoRA fine-tuning mainly enhances response initiation and style without substantial knowledge enhancement.
* Full-parameter fine-tuning tends to degrade LLM knowledge base and increase hallucination occurrences.
* Popular other methods and adjustments fail to significantly outperform simple LoRA fine-tuned models in terms of conversational quality and accuracy.

Congrats to the authors @Sreyan88 and others for their work!

Paper: https://huggingface.co/papers/2402.05119
👍
Hopkinx osanseviero ioanacroi kaleinaNyan clem Felladrin RamaniD sonalkum TheDetective Sreyan88 Arjun4707 TerraNull schoolta
/avatars/e23e00fa020ae3f3f304fbf9fb2dc244.svg
mohd khazli aswad
khazli
user
https://cdn-avatars.huggingface.co/v1/production/uploads/6454aff9273f649830234978/cvVV08YHJpJx9xWVZqgVW.jpeg
Victor Nogueira
Felladrin
user