Update README.md
Browse files
README.md
CHANGED
@@ -12,7 +12,9 @@ language:
|
|
12 |
---
|
13 |
![image/png](Pantheon.png)
|
14 |
# Pantheon-RP-1.6-12b-Nemo-KTO
|
15 |
-
Welcome to the next iteration of my Pantheon model series, in which I strive to introduce a whole collection of diverse personas that can be summoned with a simple activation phrase.
|
|
|
|
|
16 |
|
17 |
**KTO Edition:** This is an improved version of 1.6 in which I applied KTO preference training to further refine, deslopify and diversify the model's responses. Moving forward, future versions will have this type of additional training by default but for now the two 1.6 versions will live side by side.
|
18 |
|
@@ -38,13 +40,11 @@ Just like 1.5, I used a multi-stage finetuning process as Mistral Nemo was provi
|
|
38 |
|
39 |
## Inference
|
40 |
|
41 |
-
Nemo is a somewhat strange model when it comes to temperatures so I highly encourage you to experiment to see which works best.
|
42 |
```
|
43 |
-
"temperature": 0.
|
44 |
"repetition_penalty": 1.05,
|
45 |
-
"
|
46 |
-
"top_k": 40
|
47 |
-
"min_p": 0.05
|
48 |
```
|
49 |
|
50 |
Besides the basic instructional sets all other datasets were trained with character names added. Enable this at all times for an optimal experience.
|
|
|
12 |
---
|
13 |
![image/png](Pantheon.png)
|
14 |
# Pantheon-RP-1.6-12b-Nemo-KTO
|
15 |
+
Welcome to the next iteration of my Pantheon model series, in which I strive to introduce a whole collection of diverse personas that can be summoned with a simple activation phrase.
|
16 |
+
|
17 |
+
Pantheon's purpose is two-fold, as these personalities similarly enhance the general roleplay experience, helping to encompass personality traits, accents and mannerisms that language models might otherwise find difficult to convey well.
|
18 |
|
19 |
**KTO Edition:** This is an improved version of 1.6 in which I applied KTO preference training to further refine, deslopify and diversify the model's responses. Moving forward, future versions will have this type of additional training by default but for now the two 1.6 versions will live side by side.
|
20 |
|
|
|
40 |
|
41 |
## Inference
|
42 |
|
43 |
+
Nemo is a somewhat strange model when it comes to temperatures so I highly encourage you to experiment to see which works best. Here's my current preset:
|
44 |
```
|
45 |
+
"temperature": 0.8,
|
46 |
"repetition_penalty": 1.05,
|
47 |
+
"min_p": 0.025
|
|
|
|
|
48 |
```
|
49 |
|
50 |
Besides the basic instructional sets all other datasets were trained with character names added. Enable this at all times for an optimal experience.
|