
# Quantized version of meta-llama/LlamaGuard-7b

## Model Description

The model [meta-llama/LlamaGuard-7b](https://huggingface.co/meta-llama/LlamaGuard-7b) was quantized to 4bit, group_size 128, and act-order=True with auto-gptq integration in transformers (https://huggingface.co/blog/gptq-integration).

## Evaluation

To evaluate the qunatized model and compare it with the full precision model, I performed binary classification on the "toxicity" label from the ~5k samples test set of lmsys/toxic-chat.

📊 Full Precision Model:

Average Precision Score: 0.3625

📊 4-bit Quantized Model:

Average Precision Score: 0.3450


{"language": ["en"], "license": "llama2", "library_name": "transformers", "tags": ["facebook", "meta", "pytorch", "llama", "llama-2", "4bit", "gptq"], "base_model": "meta-llama/LlamaGuard-7b", "inference": false}
transformers safetensors llama text-generation facebook meta pytorch llama-2 4bit gptq conversational en base_model:meta-llama/LlamaGuard-7b license:llama2 autotrain_compatible text-generation-inference 4-bit region:us