ColdBrew-Calypso
ColdBrew-Calypso is a merge of the following models using LazyMergekit:
𧩠Configuration
const_tag:
- &SFU 0.7071067812 # Upper scaling factor (1/sqrt(2))
- &SFL 0.85 # Lower scaling factor
- &SFMLP 0.8 # MLP scaling factor
attenuate-env-lower: &attenuated_env_lower
parameters:
scale:
- filter: q_proj
value: *SFL
- filter: k_proj
value: *SFL
- filter: v_proj
value: *SFL
- filter: o_proj
value: *SFL
- filter: gate_proj
value: *SFMLP
- filter: up_proj
value: *SFMLP
- filter: down_proj
value: *SFMLP
attenuate-env-upper: &attenuated_env_upper
parameters:
scale:
- filter: q_proj
value: *SFU
- filter: k_proj
value: *SFU
- filter: v_proj
value: *SFU
- filter: o_proj
value: *SFU
- filter: gate_proj
value: *SFMLP
- filter: up_proj
value: *SFMLP
- filter: down_proj
value: *SFMLP
slices:
# Preserve input layers
- sources:
- model: SvalTek/ColdBrew-Aphid
layer_range: [0, 12]
# Early Expansion (Duplicate Twice)
- sources:
- model: SvalTek/ColdBrew-Aphid
layer_range: [13, 15]
- sources:
- model: SvalTek/ColdBrew-Aphid
layer_range: [13, 15]
<<: *attenuated_env_lower
- sources:
- model: SvalTek/ColdBrew-Aphid
layer_range: [13, 15]
<<: *attenuated_env_lower
# Late Expansion (Duplicate Twice)
- sources:
- model: SvalTek/ColdBrew-Aphid
layer_range: [15, 18]
- sources:
- model: SvalTek/ColdBrew-Aphid
layer_range: [15, 18]
<<: *attenuated_env_upper
- sources:
- model: SvalTek/ColdBrew-Aphid
layer_range: [15, 18]
<<: *attenuated_env_upper
# Preserve output layers
- sources:
- model: SvalTek/ColdBrew-Aphid
layer_range: [19, 27]
merge_method: passthrough
dtype: bfloat16
parameters:
normalize: true
int8_mask: true
π» Usage
!pip install -qU transformers accelerate
from transformers import AutoTokenizer
import transformers
import torch
model = "SvalTek/ColdBrew-Calypso"
messages = [{"role": "user", "content": "What is a large language model?"}]
tokenizer = AutoTokenizer.from_pretrained(model)
prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
pipeline = transformers.pipeline(
"text-generation",
model=model,
torch_dtype=torch.float16,
device_map="auto",
)
outputs = pipeline(prompt, max_new_tokens=256, do_sample=True, temperature=0.7, top_k=50, top_p=0.95)
print(outputs[0]["generated_text"])
- Downloads last month
- 9
Model tree for SvalTek/ColdBrew-Calypso
Unable to build the model tree, the base model loops to the model itself. Learn more.