YAML Metadata Warning: empty or missing yaml metadata in repo card (https://huggingface.co/docs/hub/model-cards#model-card-metadata)

sudo docker run --rm    \
-p 8080:80     \
-e GPTQ_BITS=4     \
-e GPTQ_GROUPSIZE=128     \
-e MAX_BEST_OF=1 \
-e MAX_BATCH_PREFILL_TOKENS=2048 \
--gpus '"device=0"'     \
-v $PWD/data:/data ghcr.io/huggingface/text-generation-inference:sha-bce5e22     \
--model-id /data/WizardCoder-Python-34B-V1.0-GPTQ \
--quantize gptq

Downloads last month: 13

Safetensors

Model size

4.98B params

Tensor type

I32

FP16

Inference Providers NEW

Text Generation

This model is not currently available via any of the supported third-party Inference Providers, and the model is not deployed on the HF Inference API.