text
Introducing UNA-ThePitbull Series
Introducing UNA-ThePitbull Series
new_line
new_line
text
We are happy to announce the release of our latest model UNA-ThePitbull, the most powerful model below 70B in the industry. In this new generation, inspired on our previous Beagle series we curated a model that balance nicely EQ and IQ. It was trained with some of the latest datasets including:
We are happy to announce the release of our latest model UNA-ThePitbull, the most powerful model below 70B in the industry. In this new generation, inspired on our previous Beagle series we curated a model that balance nicely EQ and IQ. It was trained with some of the latest datasets including:
new_line
text
* Replete-AI/code_bagel_hermes-2.5
* Replete-AI/code_bagel_hermes-2.5
new_line
text
* mlabonne/orpo-dpo-mix-40k
* mlabonne/orpo-dpo-mix-40k
new_line
text
* jondurbin/py-dpo-v0.1
* jondurbin/py-dpo-v0.1
new_line
text
Available in the hub 
Available in the hub 
resource
https://huggingface.co/fblgit/UNA-ThePitbull-21.4B-v2
model
fblgit/UNA-ThePitbull-21.4B-v2
https://huggingface.co/fblgit/UNA-ThePitbull-21.4B-v2
text
 and you can grab Quant versions sponsored by 
 and you can grab Quant versions sponsored by 
mention
@bartowski
bartowski
text
 at 
 at 
resource
https://huggingface.co/bartowski/UNA-ThePitbull-21.4B-v2-GGUF
model
bartowski/UNA-ThePitbull-21.4B-v2-GGUF
https://huggingface.co/bartowski/UNA-ThePitbull-21.4B-v2-GGUF
text
 fully compatible with Ollama, llama.cpp, etc.
 fully compatible with Ollama, llama.cpp, etc.
new_line
new_line
text
UNA
UNA
new_line
text
In this case we tried something new by alternating uniformity across layers of both MLP & Attention reducing computational requirements while keep a high performant result.
In this case we tried something new by alternating uniformity across layers of both MLP & Attention reducing computational requirements while keep a high performant result.
new_line
new_line
text
We trained him under these terms:
We trained him under these terms:
new_line
text
* ThePitbull-v1 as base: SFT maxLR 1e-4 minLR 5e-5 for 1 Epoch
* ThePitbull-v1 as base: SFT maxLR 1e-4 minLR 5e-5 for 1 Epoch
new_line
text
* DPO maxLR 1e-4 minLR 5e-5 for 1 Epoch
* DPO maxLR 1e-4 minLR 5e-5 for 1 Epoch
new_line
text
You can continue the training by merely using 5e-5 maxLR and 0 warmup steps, it should minimize catastrophic forgetting of the model.
You can continue the training by merely using 5e-5 maxLR and 0 warmup steps, it should minimize catastrophic forgetting of the model.
new_line
new_line
text
Remember if you do so, please include a Pitbull picture on your model and cite :) Have fun!
Remember if you do so, please include a Pitbull picture on your model and cite :) Have fun!
new_line
Introducing UNA-ThePitbull Series

We are happy to announce the release of our latest model UNA-ThePitbull, the most powerful model below 70B in the industry. In this new generation, inspired on our previous Beagle series we curated a model that balance nicely EQ and IQ. It was trained with some of the latest datasets including:
* Replete-AI/code_bagel_hermes-2.5
* mlabonne/orpo-dpo-mix-40k
* jondurbin/py-dpo-v0.1
Available in the hub https://huggingface.co/fblgit/UNA-ThePitbull-21.4B-v2 and you can grab Quant versions sponsored by @bartowski at https://huggingface.co/bartowski/UNA-ThePitbull-21.4B-v2-GGUF fully compatible with Ollama, llama.cpp, etc.

UNA
In this case we tried something new by alternating uniformity across layers of both MLP & Attention reducing computational requirements while keep a high performant result.

We trained him under these terms:
* ThePitbull-v1 as base: SFT maxLR 1e-4 minLR 5e-5 for 1 Epoch
* DPO maxLR 1e-4 minLR 5e-5 for 1 Epoch
You can continue the training by merely using 5e-5 maxLR and 0 warmup steps, it should minimize catastrophic forgetting of the model.

Remember if you do so, please include a Pitbull picture on your model and cite :) Have fun!

🔥
bartowski Tonic Yhyu13
🚀
thesven