File size: 4,678 Bytes
8b37e57 59f5896 62a64dc ff4415f c0e38ef ff4415f 5086543 67cd116 8b37e57 62a64dc d860812 9e66372 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 |
---
license: llama3.1
language:
- en
quantized_by: TheMelonGod
pipeline_tag: text-generation
tags:
- quantized
- safetensors
- exllamav2
base_model:
- ArliAI/Llama-3.1-8B-ArliAI-RPMax-v1.3
base_model_relation: quantized
---
**Orignal Model by:** [Arli AI](https://huggingface.co/ArliAI)
**Orignal Model:** [Llama-3.1-8B-ArliAI-RPMax-v1.3-exl2](https://huggingface.co/ArliAI/Llama-3.1-8B-ArliAI-RPMax-v1.3)
For more information about the model, I highly recommend checking out the original model page and the creator while you're at it.
**ExLlamaV2 Quantizations:**
**8.0bpw**: [8hb](https://huggingface.co/TheMelonGod/Llama-3.1-8B-ArliAI-RPMax-v1.3-exl2/tree/8hb-8.0bpw) | [6hb](https://huggingface.co/TheMelonGod/Llama-3.1-8B-ArliAI-RPMax-v1.3-exl2/tree/8.0bpw)
**7.5bpw**: [8hb](https://huggingface.co/TheMelonGod/Llama-3.1-8B-ArliAI-RPMax-v1.3-exl2/tree/8hb-7.5bpw) | [6hb](https://huggingface.co/TheMelonGod/Llama-3.1-8B-ArliAI-RPMax-v1.3-exl2/tree/7.5bpw)
**7.0bpw**: [8hb](https://huggingface.co/TheMelonGod/Llama-3.1-8B-ArliAI-RPMax-v1.3-exl2/tree/8hb-7.0bpw) | [6hb](https://huggingface.co/TheMelonGod/Llama-3.1-8B-ArliAI-RPMax-v1.3-exl2/tree/7.0bpw)
**6.5bpw**: [8hb](https://huggingface.co/TheMelonGod/Llama-3.1-8B-ArliAI-RPMax-v1.3-exl2/tree/8hb-6.5bpw) | [6hb](https://huggingface.co/TheMelonGod/Llama-3.1-8B-ArliAI-RPMax-v1.3-exl2/tree/6.5bpw)
**6.0bpw**: [8hb](https://huggingface.co/TheMelonGod/Llama-3.1-8B-ArliAI-RPMax-v1.3-exl2/tree/8hb-6.0bpw) | [6hb](https://huggingface.co/TheMelonGod/Llama-3.1-8B-ArliAI-RPMax-v1.3-exl2/tree/6.0bpw)
**5.5bpw**: [8hb](https://huggingface.co/TheMelonGod/Llama-3.1-8B-ArliAI-RPMax-v1.3-exl2/tree/8hb-5.5bpw) | [6hb](https://huggingface.co/TheMelonGod/Llama-3.1-8B-ArliAI-RPMax-v1.3-exl2/tree/5.5bpw)
**5.0bpw**: [8hb](https://huggingface.co/TheMelonGod/Llama-3.1-8B-ArliAI-RPMax-v1.3-exl2/tree/8hb-5.0bpw) | [6hb](https://huggingface.co/TheMelonGod/Llama-3.1-8B-ArliAI-RPMax-v1.3-exl2/tree/5.0bpw)
**4.5bpw**: [8hb](https://huggingface.co/TheMelonGod/Llama-3.1-8B-ArliAI-RPMax-v1.3-exl2/tree/8hb-4.5bpw) | [6hb](https://huggingface.co/TheMelonGod/Llama-3.1-8B-ArliAI-RPMax-v1.3-exl2/tree/4.5bpw)
**4.25bpw**: [8hb](https://huggingface.co/TheMelonGod/Llama-3.1-8B-ArliAI-RPMax-v1.3-exl2/tree/8hb-4.25bpw) | [6hb](https://huggingface.co/TheMelonGod/Llama-3.1-8B-ArliAI-RPMax-v1.3-exl2/tree/6hb-4.25bpw)
**4.0bpw**: [8hb](https://huggingface.co/TheMelonGod/Llama-3.1-8B-ArliAI-RPMax-v1.3-exl2/tree/8hb-4.0bpw) | [6hb](https://huggingface.co/TheMelonGod/Llama-3.1-8B-ArliAI-RPMax-v1.3-exl2/tree/4.0bpw)
**3.75bpw**: [8hb](https://huggingface.co/TheMelonGod/Llama-3.1-8B-ArliAI-RPMax-v1.3-exl2/tree/8hb-3.75bpw) | [6hb](https://huggingface.co/TheMelonGod/Llama-3.1-8B-ArliAI-RPMax-v1.3-exl2/tree/6hb-3.75bpw)
**3.5bpw**: [8hb](https://huggingface.co/TheMelonGod/Llama-3.1-8B-ArliAI-RPMax-v1.3-exl2/tree/8hb-3.5bpw) | [6hb](https://huggingface.co/TheMelonGod/Llama-3.1-8B-ArliAI-RPMax-v1.3-exl2/tree/3.5bpw)
**3.0bpw**: [8hb](https://huggingface.co/TheMelonGod/Llama-3.1-8B-ArliAI-RPMax-v1.3-exl2/tree/8hb-3.0bpw) | [6hb](https://huggingface.co/TheMelonGod/Llama-3.1-8B-ArliAI-RPMax-v1.3-exl2/tree/3.0bpw)
**2.75bpw**: [8hb](https://huggingface.co/TheMelonGod/Llama-3.1-8B-ArliAI-RPMax-v1.3-exl2/tree/8hb-2.75bpw) | [6hb](https://huggingface.co/TheMelonGod/Llama-3.1-8B-ArliAI-RPMax-v1.3-exl2/tree/6hb-2.75bpw)
**2.5bpw**: [8hb](https://huggingface.co/TheMelonGod/Llama-3.1-8B-ArliAI-RPMax-v1.3-exl2/tree/8hb-2.5bpw) | [6hb](https://huggingface.co/TheMelonGod/Llama-3.1-8B-ArliAI-RPMax-v1.3-exl2/tree/2.5bpw)
**2.25bpw**: [8hb](https://huggingface.co/TheMelonGod/Llama-3.1-8B-ArliAI-RPMax-v1.3-exl2/tree/8hb-2.25bpw) | [6hb](https://huggingface.co/TheMelonGod/Llama-3.1-8B-ArliAI-RPMax-v1.3-exl2/tree/6hb-2.25bpw)
**2.0bpw**: [8hb](https://huggingface.co/TheMelonGod/Llama-3.1-8B-ArliAI-RPMax-v1.3-exl2/tree/8hb-2.0bpw) | [6hb](https://huggingface.co/TheMelonGod/Llama-3.1-8B-ArliAI-RPMax-v1.3-exl2/tree/2.0bpw)
[Measurement File](https://huggingface.co/TheMelonGod/Llama-3.1-8B-ArliAI-RPMax-v1.3-exl2/blob/main/Llama-3.1-8B-ArliAI-RPMax-v1.3-measurement.json) _(Default/built-in calibration dataset was used)_
This is my first model quantization! If you need a specific model quantized or particular bits per weight, please let me know. I’m happy to help.
Your feedback and suggestions are always welcome! They help me improve and make quantizations better for everyone.
Special thanks to [turboderp](https://huggingface.co/turboderp) for developing the tools that made these quantizations possible. Your contributions are greatly appreciated! |