BusRune commited on
Commit
0b829ad
·
verified ·
1 Parent(s): 1c76549

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +84 -3
README.md CHANGED
@@ -1,3 +1,84 @@
1
- ---
2
- license: llama2
3
- ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ datasets:
3
+ - vicgalle/worldsim-claude-opus
4
+ - macadeliccc/opus_samantha
5
+ - anthracite-org/kalo-opus-instruct-22k-no-refusal
6
+ - lodrick-the-lafted/Sao10K_Claude-3-Opus-Instruct-9.5K-ShareGPT
7
+ - lodrick-the-lafted/Sao10K_Claude-3-Opus-Instruct-3.3K
8
+ - QuietImpostor/Sao10K-Claude-3-Opus-Instruct-15K-ShareGPT
9
+ - ChaoticNeutrals/Luminous_Opus
10
+ - kalomaze/Opus_Instruct_3k
11
+ - kalomaze/Opus_Instruct_25k
12
+ language:
13
+ - en
14
+ base_model:
15
+ - codellama/CodeLlama-13b-Instruct-hf
16
+ pipeline_tag: text-generation
17
+ license: llama2
18
+ ---
19
+
20
+ ![CL-13B-Fabula](https://files.catbox.moe/829yml.jpeg)
21
+
22
+ # CL-13B-Fabula
23
+
24
+ CL-13B-Fabula is a fine-tuned version of Facebook's CodeLlama 13B Instruct model, specifically optimized for roleplay and general knowledge tasks while maintaining its chat understanding capabilities.
25
+ This model is basically a bigger version of [L3.1-8B-Fabula](https://hf.co/BusRune/L3.1-8B-Fabula), since I wanted to make something a bit bigger but my VPS storage can't handle a 70B models so 13B model it is.
26
+
27
+ ## Model Details
28
+
29
+ - **Base Model**: [CodeLlama-13b-Instruct-hf](https://hf.co/codellama/CodeLlama-13b-Instruct-hf)
30
+ - **Chat Template**: ChatML
31
+ - **Max Input Tokens**: 16,384
32
+ - **Datasets Used In Fine-tuning:**
33
+ * [vicgalle/worldsim-claude-opus](https://hf.co/datasets/vicgalle/worldsim-claude-opus)
34
+ * [macadeliccc/opus_samantha](https://hf.co/datasets/macadeliccc/opus_samantha)
35
+ * [anthracite-org/kalo-opus-instruct-22k-no-refusal](https://hf.co/datasets/anthracite-org/kalo-opus-instruct-22k-no-refusal)
36
+ * [lodrick-the-lafted/Sao10K_Claude-3-Opus-Instruct-9.5K-ShareGPT](https://hf.co/datasets/lodrick-the-lafted/Sao10K_Claude-3-Opus-Instruct-9.5K-ShareGPT)
37
+ * [lodrick-the-lafted/Sao10K_Claude-3-Opus-Instruct-3.3K](https://hf.co/datasets/lodrick-the-lafted/Sao10K_Claude-3-Opus-Instruct-3.3K)
38
+ * [QuietImpostor/Sao10K-Claude-3-Opus-Instruct-15K-ShareGPT](https://hf.co/datasets/QuietImpostor/Sao10K-Claude-3-Opus-Instruct-15K-ShareGPT)
39
+ * [ChaoticNeutrals/Luminous_Opus](https://hf.co/datasets/ChaoticNeutrals/Luminous_Opus)
40
+ * [kalomaze/Opus_Instruct_3k](https://hf.co/datasets/kalomaze/Opus_Instruct_3k)
41
+ * [kalomaze/Opus_Instruct_25k](https://hf.co/datasets/kalomaze/Opus_Instruct_25k)
42
+
43
+
44
+ ## Chat Template
45
+ - In the finetuning ChatML were used.
46
+ ```js
47
+ function chatml2(messages) {
48
+ /**
49
+ * @param {Array<{role: string, name: string, content: string}>} messages
50
+ * @returns {{prompt: string, stop: string}}
51
+ * @description Formats messages into ChatML template format
52
+ */
53
+ const isLastMessageAssistant = messages[messages.length - 1]?.role === "assistant";
54
+
55
+ return {
56
+ prompt: messages.map((message, index) => {
57
+ const nameStr = message.name ? ` [${message.name}]` : "";
58
+ const isLast = index === messages.length - 1;
59
+ const needsEndTag = !isLastMessageAssistant || !isLast;
60
+
61
+ return `<|im_start|>${message.role.toLowerCase()}${nameStr}\n${message.content}${needsEndTag ? "<|im_end|>" : ""}`;
62
+ }).join("\n") + (isLastMessageAssistant ? "" : "\n<|im_start|>assistant\n"),
63
+ stop: "<|im_end|>"
64
+ };
65
+ }
66
+ ```
67
+
68
+ I would highly recommend you add rules as assistant role before sending to generation like this below:
69
+ ```md
70
+ <rules for="{{char}}'s responses">
71
+ 1. I will write a response as {{char}} in a short manner and will keep it detailed (I will try to keep it under 300 characters).
72
+
73
+ 2. Response formatting:
74
+ "This is for talking"
75
+ *This is for doing an action/ or self-reflection if I decide to write {{char}}'s response in first-person*
76
+ ex: "Hello, there!" *{name} waves,* "How are you doing today?"
77
+
78
+ 3. When I feel like it is needed for {{user}} to talk, I will not act as {{user}} or for them, I will simply stop generating more text via executing my EOS (end-of-string) token "<|im_end|>", to let the user write their response as {{user}}
79
+
80
+ 4. I will use my past messages as an example of how {{char}} speaks
81
+ </rules>
82
+ **{{char}}'s response:**
83
+
84
+ ```