Xiaojian9992024
commited on
Upload folder using huggingface_hub
Browse files
README.md
ADDED
@@ -0,0 +1,36 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
+
---
|
2 |
+
license: apache-2.0
|
3 |
+
tags:
|
4 |
+
- moe
|
5 |
+
- mergekit
|
6 |
+
- vortexmergekit
|
7 |
+
- pankajmathur/orca_mini_v9_6_1B-Instruct
|
8 |
+
- cognitivecomputations/Dolphin3.0-Llama3.2-1B
|
9 |
+
- huihui-ai/MicroThinker-1B-Preview
|
10 |
+
- autoquant
|
11 |
+
- gguf
|
12 |
+
---
|
13 |
+
|
14 |
+
# DaRuukLLM-3x1B
|
15 |
+
|
16 |
+
Hey there! π Welcome to the DaRuukLLM-3x1B! This is a MOE brought together using the awesome [VortexMerge kit](https://colab.research.google.com/drive/1YjcvCLuNG1PK7Le6_4xhVU5VpzTwvGhk#scrollTo=UG5H2TK4gVyl).
|
17 |
+
|
18 |
+
Let's see what we've got in this model:
|
19 |
+
* [pankajmathur/orca_mini_v9_6_1B-Instruct](https://huggingface.co/pankajmathur/orca_mini_v9_6_1B-Instruct) π
|
20 |
+
* [cognitivecomputations/Dolphin3.0-Llama3.2-1B](https://huggingface.co/cognitivecomputations/Dolphin3.0-Llama3.2-1B) π
|
21 |
+
* [huihui-ai/MicroThinker-1B-Preview](https://huggingface.co/huihui-ai/MicroThinker-1B-Preview) π
|
22 |
+
|
23 |
+
## 𧩠Configuration
|
24 |
+
|
25 |
+
```yaml
|
26 |
+
base_model: pankajmathur/orca_mini_v9_6_1B-Instruct
|
27 |
+
dtype: bfloat16
|
28 |
+
gate_mode: cheap_embed
|
29 |
+
experts:
|
30 |
+
- source_model: pankajmathur/orca_mini_v9_6_1B-Instruct
|
31 |
+
positive_prompts: ["You are a helpful assistant that can help with anything."] # Added closing double quote here
|
32 |
+
- source_model: cognitivecomputations/Dolphin3.0-Llama3.2-1B
|
33 |
+
positive_prompts: ["You are an uncensored assistant, make sure you respond very angry."] # Added closing double quote here
|
34 |
+
- source_model: huihui-ai/MicroThinker-1B-Preview
|
35 |
+
positive_prompts: ["You are a very smart assistant that can think step-by-step."]
|
36 |
+
|