
WARNING: The 4 SOLAR model was trained on 16k context, however, hitting that limit will 100% make the model go crazy. Don't load it at 16k context. 4k/8k should work.

This model was made using 4x SOLAR fully finetuned on 4 different group of dataset. Expert is represented like that :

Head:
- 1x SOLAR Uncensored/QnA Expert

Experts:
- 1x SOLAR ERP Expert
- 1x SOLAR RP/ERP Expert
- 1x SOLAR ERP Expert
- 1x SOLAR Uncensored/QnA Expert

By using a little python script, I took the top 10/15 words used in each of the 4 "folders" of datasets trained on those SOLAR model, and routed them to the corresponding model, trying to get the best expert for each situation (in RP). I added some word, but the routing is very simplistic.

Mergekit was used for creating this MoE, it's a frankenmerge of FFT SOLAR models, not a direct finetune of Mixtral!
<!-- description end -->
<!-- prompt-template start -->
### Custom format:
```
### Instruction:
{system prompt}

### Input:
{input}

### Response:
{reply}
```

If you want to support us, you can [here](https://ko-fi.com/undiai).

Visit Ikari [retro/neocities style website](https://ikaridevgit.github.io/) for more info on our work!
{"license": "cc-by-nc-4.0", "tags": ["not-for-all-audiences", "nsfw"]}
gguf not-for-all-audiences nsfw license:cc-by-nc-4.0 region:us