text
Low Rank Adaptation (LoRA) is a popular method for fine-tuning but why...? 🤔
Low Rank Adaptation (LoRA) is a popular method for fine-tuning but why...? 🤔
new_line
new_line
text
LoRA is the most widely adopted method for Parameter Efficient Fine-Tuning (PEFT) of Large Language Models (LLMs) 🤖. It reduces the number of trainable parameters and memory usage while achieving comparable performance to full fine-tuning 💾.
LoRA is the most widely adopted method for Parameter Efficient Fine-Tuning (PEFT) of Large Language Models (LLMs) 🤖. It reduces the number of trainable parameters and memory usage while achieving comparable performance to full fine-tuning 💾.
new_line
new_line
text
Good folks at 
Good folks at 
mention
@predibase
predibase
text
  have released LoRA Land! 🌍
  have released LoRA Land! 🌍
new_line
new_line
text
A research paper where they measure the quality of LLMs fine-tuned with quantized low-rank adapters across 10 base models and 31 tasks for a total of a whopping 310 models 📊.
A research paper where they measure the quality of LLMs fine-tuned with quantized low-rank adapters across 10 base models and 31 tasks for a total of a whopping 310 models 📊.
new_line
new_line
text
According to the paper, 4-bit LoRA fine-tuned models outperform base models by 34 points 🔝 and GPT-4 by 10 points on average 👏.
According to the paper, 4-bit LoRA fine-tuned models outperform base models by 34 points 🔝 and GPT-4 by 10 points on average 👏.
new_line
new_line
text
Here comes the best part... 🥁
Here comes the best part... 🥁
new_line
new_line
text
They have also evaluated the latency and concurrency capabilities of LoRAX, an open-source Multi-LoRA inference server that facilitates the deployment of multiple LoRA fine-tuned models on a single GPU using the shared base model weights and dynamic adapter loading 🚀.
They have also evaluated the latency and concurrency capabilities of LoRAX, an open-source Multi-LoRA inference server that facilitates the deployment of multiple LoRA fine-tuned models on a single GPU using the shared base model weights and dynamic adapter loading 🚀.
new_line
new_line
text
And did I mention LoRAX is completely open-source! 🎉
And did I mention LoRAX is completely open-source! 🎉
new_line
new_line
text
Paper: 
Paper: 
link
https://arxiv.org/pdf/2405.00732
https://arxiv.org/pdf/2405.00732
new_line
new_line
text
While the claims are lofty and yet to be vetted, this is amazing for serving fine-tuned models without breaking the bank! 💸
While the claims are lofty and yet to be vetted, this is amazing for serving fine-tuned models without breaking the bank! 💸
new_line
new_line
text
PS: 🤯They got LoRAX to host 25 LoRA fine-tuned Mistral-7B LLMs on a single NVIDIA A100 GPU with 80GB memory 💻.
PS: 🤯They got LoRAX to host 25 LoRA fine-tuned Mistral-7B LLMs on a single NVIDIA A100 GPU with 80GB memory 💻.
new_line
new_line
resource
https://huggingface.co/papers/2405.00732
paper
2405.00732
https://huggingface.co/papers/2405.00732
LoRA Land: 310 Fine-tuned LLMs that Rival GPT-4, A Technical Report (2405.00732)
Low Rank Adaptation (LoRA) is a popular method for fine-tuning but why...? 🤔

LoRA is the most widely adopted method for Parameter Efficient Fine-Tuning (PEFT) of Large Language Models (LLMs) 🤖. It reduces the number of trainable parameters and memory usage while achieving comparable performance to full fine-tuning 💾.

Good folks at @predibase  have released LoRA Land! 🌍

A research paper where they measure the quality of LLMs fine-tuned with quantized low-rank adapters across 10 base models and 31 tasks for a total of a whopping 310 models 📊.

According to the paper, 4-bit LoRA fine-tuned models outperform base models by 34 points 🔝 and GPT-4 by 10 points on average 👏.

Here comes the best part... 🥁

They have also evaluated the latency and concurrency capabilities of LoRAX, an open-source Multi-LoRA inference server that facilitates the deployment of multiple LoRA fine-tuned models on a single GPU using the shared base model weights and dynamic adapter loading 🚀.

And did I mention LoRAX is completely open-source! 🎉

Paper: https://arxiv.org/pdf/2405.00732

While the claims are lofty and yet to be vetted, this is amazing for serving fine-tuned models without breaking the bank! 💸

PS: 🤯They got LoRAX to host 25 LoRA fine-tuned Mistral-7B LLMs on a single NVIDIA A100 GPU with 80GB memory 💻.

https://huggingface.co/papers/2405.00732