transformers pytorch mixtral text-generation license:cc-by-nc-4.0 autotrain_compatible endpoints_compatible text-generation-inference region:us
---
license: cc-by-nc-4.0
---

# Noromaid-v0.1-mixtral-8x7b-v3 4bpw
Exllama quant of [NeverSleep/Noromaid-v0.1-mixtral-8x7b-v3](https://huggingface.co/NeverSleep/Noromaid-v0.1-mixtral-8x7b-v3)

You probably want to use the [3.5bpw](https://huggingface.co/Kooten/Noromaid-v0.1-mixtral-8x7b-v3-3.5bpw-exl2) version, it lands very close to 24gb of vram with 8bit cache enabled. GGUF is probably preferable if you have less than 24gb.

Other BPW's [2.7bpw](https://huggingface.co/Kooten/Noromaid-v0.1-mixtral-8x7b-v3-2.7bpw-exl2), [3.0bpw](https://huggingface.co/Kooten/Noromaid-v0.1-mixtral-8x7b-v3-3bpw-exl2), [3.5bpw](https://huggingface.co/Kooten/Noromaid-v0.1-mixtral-8x7b-v3-3.5bpw-exl2), [4.0bpw](https://huggingface.co/Kooten/Noromaid-v0.1-mixtral-8x7b-v3-4bpw-exl2)

## Prompt format:

```
### Instruction:
{system prompt}

### Input:
{input}

### Response:
{reply}
```
### Sillytavern presets
[Discussion with presets](https://huggingface.co/NeverSleep/Noromaid-v0.1-mixtral-8x7b-v3/discussions/1)
- Context : https://files.catbox.moe/vnnsra.json
- Instruct : https://files.catbox.moe/7sudj5.json
- Preset : https://files.catbox.moe/og94e5.json

### Contact
Kooten on discord.
