---
license: apache-2.0
language:
- en
base_model:
- Qwen/QwQ-32B-Preview
pipeline_tag: text-generation
tags:
- chat
- qwen2
---
# QwQ-32B-Preview-bnb-4bit

## Introduction

QwQ-32B-Preview-bnb-4bit is a 4-bit quantized version of the [QwQ-32B-Preview](https://huggingface.co/Qwen/QwQ-32B-Preview) model, utilizing the Bits and Bytes (bnb) quantization technique. This quantization significantly reduces the model's size and inference latency, making it more accessible for deployment on resource-constrained hardware.

## Model Details

- **Quantization:** 4-bit using Bits and Bytes (bnb)
- **Base Model:** [Qwen/QwQ-32B-Preview](https://huggingface.co/Qwen/QwQ-32B-Preview)
- **Parameters:** 32.5 billion
- **Context Length:** Up to 32,768 tokens