DevQuasar/prithivMLmods.Triangulum-5B-it-GGUF
Text Generation
•
Updated
•
10
DevQuasar
community
Verified
AI & ML interests
Open-Source LLMs, Local AI Projects: https://pypi.org/project/llm-predictive-router/
Recent Activity
View all activity
DevQuasar's activity
csabakecskemeti
updated
2
models
about 3 hours ago
csabakecskemeti
updated
a
model
about 4 hours ago
csabakecskemeti
updated
a
model
about 5 hours ago
csabakecskemeti
updated
a
model
about 6 hours ago
csabakecskemeti
updated
a
model
about 7 hours ago
csabakecskemeti
updated
a
model
about 8 hours ago
csabakecskemeti
updated
2
models
about 13 hours ago
csabakecskemeti
updated
a
model
about 14 hours ago
csabakecskemeti
posted
an
update
2 days ago
Post
404
Managed to run the Q2 quantized Deepseek V3 base locally
The quants are uploading (probably ~10-12hrs) here: DevQuasar/deepseek-ai.DeepSeek-V3-Base-GGUF
The quants are uploading (probably ~10-12hrs) here: DevQuasar/deepseek-ai.DeepSeek-V3-Base-GGUF
-
- 1 reply
csabakecskemeti
posted
an
update
3 days ago
Post
509
Just wondering why the number of parameters changed in the model attributes/Model size from 685B to 684B after converting
deepseek-ai/DeepSeek-V3-Base from FP8 to BF16:
DevQuasar/deepseek-ai.DeepSeek-V3-Base-bf16
and not just for me:
opensourcerelease/DeepSeek-V3-Base-bf16
??
DevQuasar/deepseek-ai.DeepSeek-V3-Base-bf16
and not just for me:
opensourcerelease/DeepSeek-V3-Base-bf16
??
csabakecskemeti
posted
an
update
5 days ago
Post
1478
Happy New Year, Huggingface community!
In 2025, I'll continue my quantization (and some fine-tuning) efforts to support the open-source AI and Make knowledge free for everyone.
https://huggingface.co/DevQuasar
https://devquasar.com/
In 2025, I'll continue my quantization (and some fine-tuning) efforts to support the open-source AI and Make knowledge free for everyone.
https://huggingface.co/DevQuasar
https://devquasar.com/
- 1 reply
csabakecskemeti
posted
an
update
7 days ago
Post
2070
The
deepseek-ai/DeepSeek-V3-Base
model has featured today on CNBC tech news. The whale made a splash by using FP8 and shrink the cost of training significantly!
https://youtu.be/NJljq429cGk?si=kgk-ogPTMfJKsaA2
model has featured today on CNBC tech news. The whale made a splash by using FP8 and shrink the cost of training significantly!
https://youtu.be/NJljq429cGk?si=kgk-ogPTMfJKsaA2
- 3 replies
csabakecskemeti
posted
an
update
8 days ago
Post
1444
I've built a small utility to split safetensors file by file.
The issue/need came up when I've tried to convert the new Deepseek V3 model from FP8 to BF16.
The only Ada architecture GPU I have is an RTX 4080 and the 16GB vram was just wasn't enough for the conversion.
BTW: I'll upload the bf16 version here:
DevQuasar/deepseek-ai.DeepSeek-V3-Base-bf16
(it will take a while - days with my upload speed)
If anyone has access the resources to test it I'd appreciate a feedback if it's working or not.
The tool, is available from here:
https://github.com/csabakecskemeti/ai_utils/blob/main/safetensor_splitter.py
It's splitting every file to n pieces by the layers if possible, and create a new "model.safetensors.index.json" file.
I've tested it with Llama 3.1 8B and multiple split sizes, and validated by using inference pipeline.
use
Please note current version expects the model is already multiple file and have a "model.safetensors.index.json" layer-safetensor mapping file.
The issue/need came up when I've tried to convert the new Deepseek V3 model from FP8 to BF16.
The only Ada architecture GPU I have is an RTX 4080 and the 16GB vram was just wasn't enough for the conversion.
BTW: I'll upload the bf16 version here:
DevQuasar/deepseek-ai.DeepSeek-V3-Base-bf16
(it will take a while - days with my upload speed)
If anyone has access the resources to test it I'd appreciate a feedback if it's working or not.
The tool, is available from here:
https://github.com/csabakecskemeti/ai_utils/blob/main/safetensor_splitter.py
It's splitting every file to n pieces by the layers if possible, and create a new "model.safetensors.index.json" file.
I've tested it with Llama 3.1 8B and multiple split sizes, and validated by using inference pipeline.
use
--help for usagePlease note current version expects the model is already multiple file and have a "model.safetensors.index.json" layer-safetensor mapping file.
csabakecskemeti
posted
an
update
19 days ago
Post
1225
tiiuae Falcon3 10B Q8 playground:
https://huggingface.co/spaces/DevQuasar/Mi50
Also find my tiiuae Falcon3 Quant collection here:
https://huggingface.co/collections/DevQuasar/tiiuae-falcon3-676236626f3c57d1a19c6c1d
Enjoy!
https://huggingface.co/spaces/DevQuasar/Mi50
Also find my tiiuae Falcon3 Quant collection here:
https://huggingface.co/collections/DevQuasar/tiiuae-falcon3-676236626f3c57d1a19c6c1d
Enjoy!
csabakecskemeti
posted
an
update
22 days ago
Post
4492
The AMD Instinct MI50 (~$110) is surprisingly fast for inference Quantized models.
This runs a Llama 3.1 8B Q8 with Llama.cpp
https://huggingface.co/spaces/DevQuasar/Mi50
A little blogpost about the HW
http://devquasar.com/uncategorized/amd-radeon-instinct-mi50-cheap-inference/
This runs a Llama 3.1 8B Q8 with Llama.cpp
https://huggingface.co/spaces/DevQuasar/Mi50
A little blogpost about the HW
http://devquasar.com/uncategorized/amd-radeon-instinct-mi50-cheap-inference/
csabakecskemeti
posted
an
update
about 1 month ago
Post
1167
Fine Tuned a Llama3.2 3B on the MS Orca-Agents dataset for Analytical-Reasoning
r=16, Alpha=32
If you want to give it a try:
Model:
DevQuasar/analytical_reasoning_r16a32_unsloth-Llama-3.2-3B-Instruct-bnb-4bit
Adapter:
DevQuasar/analytical_reasoning_r16a32_unsloth-Llama-3.2-3B-Instruct-bnb-4bit_adapter
Quants:
DevQuasar/analytical_reasoning_r16a32_unsloth-Llama-3.2-3B-Instruct-bnb-4bit-GGUF
r=16, Alpha=32
If you want to give it a try:
Model:
DevQuasar/analytical_reasoning_r16a32_unsloth-Llama-3.2-3B-Instruct-bnb-4bit
Adapter:
DevQuasar/analytical_reasoning_r16a32_unsloth-Llama-3.2-3B-Instruct-bnb-4bit_adapter
Quants:
DevQuasar/analytical_reasoning_r16a32_unsloth-Llama-3.2-3B-Instruct-bnb-4bit-GGUF
-
- 1 reply
csabakecskemeti
posted
an
update
about 1 month ago
Post
1180
I have this small utility: no_more_typo
It is running in the background and able to call the LLM model to update the text on the clipboard. I think it would be ideal to fix typos and syntax.
I have just added the option to use custom prompt templates to perform different tasks.
Details, code and executable:
https://github.com/csabakecskemeti/no_more_typo
https://devquasar.com/no-more-typo/
It is running in the background and able to call the LLM model to update the text on the clipboard. I think it would be ideal to fix typos and syntax.
I have just added the option to use custom prompt templates to perform different tasks.
Details, code and executable:
https://github.com/csabakecskemeti/no_more_typo
https://devquasar.com/no-more-typo/
csabakecskemeti
posted
an
update
about 1 month ago
Post
292
Repurposed my older AI workstation to a homelab server, it has received 2xV100 + 1xP40
I can reach huge 210k token context size with MegaBeam-Mistral-7B-512k-GGUF ~70+tok/s, or run Llama-3.1-Nemotron-70B-Instruct-HF-GGUF with 50k Context ~10tok/s (V100 only 40k ctx and 15tok/s).
Also able to Lora finetune with similar performace as an RTX3090.
It moved to the garage to no complaints for the noise from the family. Will move to a Rack soon :D
I can reach huge 210k token context size with MegaBeam-Mistral-7B-512k-GGUF ~70+tok/s, or run Llama-3.1-Nemotron-70B-Instruct-HF-GGUF with 50k Context ~10tok/s (V100 only 40k ctx and 15tok/s).
Also able to Lora finetune with similar performace as an RTX3090.
It moved to the garage to no complaints for the noise from the family. Will move to a Rack soon :D
-
- 2 replies