
# HoloViolet-7B-test1

My first merge, trying to find a sweet spot between the schizo creativity of holodeck and the consistent quality and spatial awareness of v1olet.
It seems to achieve more coherency than base holodeck, but not consistent enough yet. Prompt format is Mistral or Roleplay on sillytavern, whichever works best for you.

HoloViolet-7B-test1 is a merge of the following models using [LazyMergekit](https://colab.research.google.com/drive/1obulZ1ROXHjYLn6PPZJwRR6GzgQogxxb?usp=sharing):
* [GreenNode/GreenNode-mini-7B-multilingual-v1olet](https://huggingface.co/GreenNode/GreenNode-mini-7B-multilingual-v1olet)
* [KoboldAI/Mistral-7B-Holodeck-1](https://huggingface.co/KoboldAI/Mistral-7B-Holodeck-1)

## 🧩 Configuration

```yaml
slices:
  - sources:
      - model: GreenNode/GreenNode-mini-7B-multilingual-v1olet
        layer_range: [0, 32]
      - model: KoboldAI/Mistral-7B-Holodeck-1
        layer_range: [0, 32]
merge_method: slerp
base_model: GreenNode/GreenNode-mini-7B-multilingual-v1olet
parameters:
  t:
    - filter: self_attn
      value: [0, 0.5, 0.3, 0.7, 1]
    - filter: mlp
      value: [1, 0.5, 0.7, 0.3, 0]
    - value: 0.5
dtype: bfloat16
```

## 💻 Usage

```python
!pip install -qU transformers accelerate

from transformers import AutoTokenizer
import transformers
import torch

model = "son-of-man/HoloViolet-7B-test1"
messages = [{"role": "user", "content": "What is a large language model?"}]

tokenizer = AutoTokenizer.from_pretrained(model)
prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
pipeline = transformers.pipeline(
    "text-generation",
    model=model,
    torch_dtype=torch.float16,
    device_map="auto",
)

outputs = pipeline(prompt, max_new_tokens=256, do_sample=True, temperature=0.7, top_k=50, top_p=0.95)
print(outputs[0]["generated_text"])
```
{"tags": ["merge", "mergekit", "lazymergekit", "GreenNode/GreenNode-mini-7B-multilingual-v1olet", "KoboldAI/Mistral-7B-Holodeck-1"], "base_model": ["GreenNode/GreenNode-mini-7B-multilingual-v1olet", "KoboldAI/Mistral-7B-Holodeck-1"]}
transformers safetensors gguf mistral text-generation merge mergekit lazymergekit GreenNode/GreenNode-mini-7B-multilingual-v1olet KoboldAI/Mistral-7B-Holodeck-1 base_model:GreenNode/GreenNode-mini-7B-multilingual-v1olet base_model:KoboldAI/Mistral-7B-Holodeck-1 autotrain_compatible endpoints_compatible text-generation-inference region:us