text
To demonstrate that it was possible, I performed a "trapezoid" gradient merge of a Llama 3 8B model onto Llama 3.1 8B Instruct, favoring the L3.1 model at the ends in order to preserve coherence and limiting the influence of the L3 model to at most 0.1 weight. Tested to 16k context length.
To demonstrate that it was possible, I performed a "trapezoid" gradient merge of a Llama 3 8B model onto Llama 3.1 8B Instruct, favoring the L3.1 model at the ends in order to preserve coherence and limiting the influence of the L3 model to at most 0.1 weight. Tested to 16k context length.
new_line
resource
https://huggingface.co/grimjim/Llama-Nephilim-Metamorphosis-v2-8B
model
grimjim/Llama-Nephilim-Metamorphosis-v2-8B
https://huggingface.co/grimjim/Llama-Nephilim-Metamorphosis-v2-8B
To demonstrate that it was possible, I performed a "trapezoid" gradient merge of a Llama 3 8B model onto Llama 3.1 8B Instruct, favoring the L3.1 model at the ends in order to preserve coherence and limiting the influence of the L3 model to at most 0.1 weight. Tested to 16k context length.
https://huggingface.co/grimjim/Llama-Nephilim-Metamorphosis-v2-8B
😎
John6666 TuringsSolutions