Airoboros c34 2.2.1 Mistral GGUF

CodeLlama 34b With Airoboros 2.2.1 dataset by Jon Durbin

Then

With Mistral AI 7b 0.1 delta bits compared to Llama2 (extracted by Undi95), merged by myself.

---

Base model (CodeLlama) training context : 16k (max context up to 96k with the base ROPE)

Mistral injection training context : 8k (Sliding Windows Attention is likely inoperant on such a merge/injection)

---

For test and amusement only.


Prompt : Airoboros
{"license": "llama2"}
gguf license:llama2 region:us