jerryzh168/Meta-Llama-3-8B-torchao-int4_weight_only-gs_128 (Quantized)
Description
This model is a quantized version of the original model jerryzh168/Meta-Llama-3-8B-torchao-int4_weight_only-gs_128. It has been quantized using int4_weight_only quantization with torchao.
Quantization Details
- Quantization Type: int4_weight_only
- Group Size: 128
Usage
You can use this model in your applications by loading it directly from the Hugging Face Hub:
from transformers import AutoModel
model = AutoModel.from_pretrained("jerryzh168/Meta-Llama-3-8B-torchao-int4_weight_only-gs_128")
- Downloads last month
- 3
Inference Providers
NEW
This model is not currently available via any of the supported third-party Inference Providers, and
HF Inference API was unable to determine this model's library.
Model tree for jerryzh168/Meta-Llama-3-8B-torchao-int4_weight_only-gs_128
Unable to build the model tree, the base model loops to the model itself. Learn more.