

# Prompt Format:

```
### Instruction: {question}

### Response: {response}
```


Use these options to load with bnb:

4-bit params: {'load_in_4bit': True, 'bnb_4bit_compute_dtype':        
                         torch.float16, 'bnb_4bit_quant_type': 'nf4', 'bnb_4bit_use_double_quant': True}
{"license": "other", "license_name": "qwen", "license_link": "LICENSE"}
transformers safetensors llama text-generation license:other autotrain_compatible endpoints_compatible text-generation-inference 4-bit region:us