transformers safetensors llama text-generation not-for-all-audiences nsfw license:cc-by-nc-4.0 autotrain_compatible endpoints_compatible text-generation-inference region:us
---
license: cc-by-nc-4.0
tags:
- not-for-all-audiences
- nsfw
---
## Exl2 version of [Undi95/Nethena-MLewd-Xwin-23B](https://huggingface.co/Undi95/Nethena-MLewd-Xwin-23B)  

## branch  
[main](https://huggingface.co/IHaBiS/Nethena-MLewd-Xwin-23B-exl2/tree/main) : 3.75bpw h8  
[b3.75h8](https://huggingface.co/IHaBiS/Nethena-MLewd-Xwin-23B-exl2/tree/b3.75h8) : 3.75bpw h8  
[b4h6](https://huggingface.co/IHaBiS/Nethena-MLewd-Xwin-23B-exl2/tree/b4h6) : 4bpw h6  
[b4h8](https://huggingface.co/IHaBiS/Nethena-MLewd-Xwin-23B-exl2/tree/b4h8) : 4bpw h8  

I checked that main branch runs on **24G GPU** (tested on Runpod 3090 server)  
Maybe I'll test 3.8bpw or 3.9bpw next time (Not sure about it)  

### below this line is original readme  


Undi doing chemistry again.

Layer of Xwin-Mlewd was added in a different way than I do before, result seem good, but I'm a VRAMlet so I can only run the Q2 at 2k context for now.

Need to see if it really work good or I was just lucky with my prompt.

OG model : [NeverSleep/Nethena-13B](https://huggingface.co/NeverSleep/Nethena-13B)

## Prompt template: Alpaca

```
Below is an instruction that describes a task. Write a response that appropriately completes the request.

### Instruction:
{prompt}

### Response:
```

LimaRP is always kicking in and thus, this can be used to have more control on the size of the output.

![image/png](https://cdn-uploads.huggingface.co/production/uploads/63ab1241ad514ca8d1430003/2ftpUX9khVcddk-VGEH3p.png)

Thanks Ikari.