Directly quantized 4bit model with `bitsandbytes`.

Unsloth can finetune LLMs with QLoRA 2.2x faster and use 62% less memory!

We have a Google Colab Tesla T4 notebook for Llama 7b here: https://colab.research.google.com/drive/1lBzz5KeZJKXjvivbYvmGarix9Ao6Wxe5?usp=sharing
{"license": "llama2"}
transformers safetensors llama text-generation license:llama2 autotrain_compatible endpoints_compatible has_space text-generation-inference region:us