text
Gemma QLoRA finetuning is now 2.4x faster and uses 58% less VRAM than FA2 through 🦥Unsloth! Had to rewrite our Cross Entropy Loss kernels to work on all vocab sizes, re-design our manual autograd engine to accept all activation functions, and more! I wrote all about our learnings in our blog post: 
Gemma QLoRA finetuning is now 2.4x faster and uses 58% less VRAM than FA2 through 🦥Unsloth! Had to rewrite our Cross Entropy Loss kernels to work on all vocab sizes, re-design our manual autograd engine to accept all activation functions, and more! I wrote all about our learnings in our blog post: 
link
https://unsloth.ai/blog/gemma
https://unsloth.ai/blog/gemma
text
.
.
new_line
new_line
text
Also have a Colab notebook with no OOMs, and has 2x faster inference for Gemma & how to merge and save to llama.cpp GGUF & vLLM: 
Also have a Colab notebook with no OOMs, and has 2x faster inference for Gemma & how to merge and save to llama.cpp GGUF & vLLM: 
link
https://colab.research.google.com/drive/10NbwlsRChbma1v55m8LAPYG15uQv6HLo?usp=sharing
https://colab.research.google.com/drive/10NbwlsRChbma1v55m8LAPYG15uQv6HLo?usp=sharing
new_line
new_line
text
And uploaded 4bit pre-quantized versions for Gemma 2b and 7b: 
And uploaded 4bit pre-quantized versions for Gemma 2b and 7b: 
resource
https://huggingface.co/unsloth/gemma-7b-bnb-4bit
model
unsloth/gemma-7b-bnb-4bit
https://huggingface.co/unsloth/gemma-7b-bnb-4bit
text
resource
https://huggingface.co/unsloth/gemma-2b-bnb-4bit
model
unsloth/gemma-2b-bnb-4bit
https://huggingface.co/unsloth/gemma-2b-bnb-4bit
new_line
new_line
code_fence
```
from unsloth import FastLanguageModel
model, tokenzer = FastLanguageModel.from_pretrained("unsloth/gemma-7b")
model = FastLanguageModel.get_peft_model(model)
```
from unsloth import FastLanguageModel
model, tokenzer = FastLanguageModel.from_pretrained("unsloth/gemma-7b")
model = FastLanguageModel.get_peft_model(model)
Gemma QLoRA finetuning is now 2.4x faster and uses 58% less VRAM than FA2 through 🦥Unsloth! Had to rewrite our Cross Entropy Loss kernels to work on all vocab sizes, re-design our manual autograd engine to accept all activation functions, and more! I wrote all about our learnings in our blog post: https://unsloth.ai/blog/gemma.

Also have a Colab notebook with no OOMs, and has 2x faster inference for Gemma & how to merge and save to llama.cpp GGUF & vLLM: https://colab.research.google.com/drive/10NbwlsRChbma1v55m8LAPYG15uQv6HLo?usp=sharing

And uploaded 4bit pre-quantized versions for Gemma 2b and 7b: https://huggingface.co/unsloth/gemma-7b-bnb-4bit https://huggingface.co/unsloth/gemma-2b-bnb-4bit

```
from unsloth import FastLanguageModel
model, tokenzer = FastLanguageModel.from_pretrained("unsloth/gemma-7b")
model = FastLanguageModel.get_peft_model(model)
```
❤️
Sylvestre harpreetsahota cnmoro clem osanseviero ybelkada rbiswasfc alielfilali01 danielus ambivalent02 victor
👍
osanseviero ybelkada sbarman25 alielfilali01
https://cdn-avatars.huggingface.co/v1/production/uploads/64aea8ff67511bd3d965697b/Jxn52EmDF5RApJh8antxn.jpeg
Feynman Innovations
ajibawa-2023
user
https://cdn-avatars.huggingface.co/v1/production/uploads/62ecdc18b72a69615d6bd857/ixLCk0TwaCVyL_nAfrgEs.png
Daniel Han-Chen
danielhanchen
user