What am i doing wrong... it's like it has braindamage
Trying to make it work but it goes off the rails in either Mistral or Metharme.
Is there anything I am missing?
I am trying this model to see if it can beat my favorite model, Luminum-v0.1-123B-exl2-4.0bpw
It does respond nicely in its first response. But.... anything long or with context and the model quickly starts to look it has brain damage on either Mistral or Metharme.
I tried a few things like limiting context to 32k, different torch/exllama/flash drivers but it all resulted in the same.
Can I perhaps ask for a example setup in how to use it in combo with SillyTavern?
Or is this just a flaw everyone is experiencing on long form like story writing in this model?
It's almost certainly a sampler settings issue. Use the ST preset found here: https://huggingface.co/Konnect1221/Methception-SillyTavern-Preset