mixtral text-generation en license:apache-2.0 region:us
---
inference: false
language:
- en
pipeline_tag: text-generation
tags:
- mixtral
license: apache-2.0
---
# Mixtral-8x7B-Instruct-v0.1-limarp-exl2

Exllama v2 quant of [Doctor-Shotgun/Mixtral-8x7B-Instruct-v0.1-limarp](https://huggingface.co/Doctor-Shotgun/Mixtral-8x7B-Instruct-v0.1-limarp)

Branches:
- main: measurement.json calculated at 2048 token calibration rows on exl2 default calibration dataset
- [3.5bpw-h6](https://huggingface.co/Doctor-Shotgun/Mixtral-8x7B-Instruct-v0.1-limarp-exl2/tree/3.5bpw-h6): 3.5 decoder bits per weight, 6 head bits
  - ideal for 24gb VRAM
- [6.0bpw-h6](https://huggingface.co/Doctor-Shotgun/Mixtral-8x7B-Instruct-v0.1-limarp-exl2/tree/6.0bpw-h6): 6 decoder bits per weight, 6 head bits
  - ideal for 48gb VRAM