unsloth
/

DeepSeek-V3-GGUF

Inference Endpoints

Model card Files Files and versions Community

Resources

View closed (2)

Core dumped when merging DeepSeek-V3-Q2_K_XS

#12 opened 2 days ago by

First review, Q5-K-M require 502Gb RAM, better than Meta 405billions

#11 opened 9 days ago by

Issue with --n-gpu-layers 5 Parameter: Model Only Running on CPU

#10 opened 10 days ago by

I’m new to GGUF quants

#9 opened 10 days ago by

I loaded DeepSeek-V3-Q5_K_M up on my 10yrs old old Tesla M40 (Dell C4130)

#8 opened 12 days ago by

why use q5 for key cache?

#7 opened 12 days ago by

What is the required GPU size to run Is a 4090 possible and does it support ollama

#5 opened 13 days ago by

I'm a newbie. How to use?

#4 opened 13 days ago by

Getting error with Q3-K-M

#2 opened 13 days ago by

Are these imatrix GGUF quants?

#1 opened 14 days ago by