Update README.md
Browse files
README.md
CHANGED
@@ -34,9 +34,9 @@ Healed Llama-3 15B Frankenmerge
|
|
34 |
|
35 |
![image/jpeg](https://cdn-uploads.huggingface.co/production/uploads/64f74b6e6389380c77562762/MCdGdalCCtOVPn8X7rqha.jpeg)
|
36 |
|
37 |
-
This is the third iteration "Emerald" of the final four and the one I liked the most. It has had limited testing though, but seems decent.
|
38 |
|
39 |
-
Findings: o_proj and down_proj can be stolen from Aethora-v2 so new 15B frankenmerges don't seem to need finetuning to "heal" the layers at all.
|
40 |
|
41 |
# <h1>merge</h1>
|
42 |
|
@@ -45,7 +45,7 @@ This is a merge of pre-trained language models created using [mergekit](https://
|
|
45 |
# <h1>Merge Details</h1>
|
46 |
# <h1>Merge Method</h1>
|
47 |
|
48 |
-
This model was merged using an iterative merging process. (Probably ~
|
49 |
|
50 |
# <h1>Models Merged</h1>
|
51 |
|
|
|
34 |
|
35 |
![image/jpeg](https://cdn-uploads.huggingface.co/production/uploads/64f74b6e6389380c77562762/MCdGdalCCtOVPn8X7rqha.jpeg)
|
36 |
|
37 |
+
This is the third iteration "Emerald" of the final four and the one I liked the most. It has had limited testing though, but seems relatively decent.
|
38 |
|
39 |
+
Findings: o_proj and down_proj can be stolen from Aethora-v2 so new 15B frankenmerges don't seem to really ***need*** finetuning to "heal" the layers at all.
|
40 |
|
41 |
# <h1>merge</h1>
|
42 |
|
|
|
45 |
# <h1>Merge Details</h1>
|
46 |
# <h1>Merge Method</h1>
|
47 |
|
48 |
+
This model was merged using an iterative merging process. (Probably ~10 models got thrown away in the process.)
|
49 |
|
50 |
# <h1>Models Merged</h1>
|
51 |
|