license:mit region:us
---
license: mit
---

# Qlora adapters for Mixtral-8x7B-v0.1-hf-4bit-mlx 

## fine-tuned on guanaco dataset

## inference vis mlx-lm
```
from mlx_lm import load, generate

model, tokenizer = load("mlx-community/Mixtral-8x7B-v0.1-hf-4bit-mlx",adapter_file="adapters.npz")

generate(model=model, tokenizer=tokenizer, prompt="### Human: write a quick sort in python.\n### Assistant: ", max_tokens=500, verbose=True,temp=0.3)
```

## serve as an API Service

```
pip install mlx-llm-server
mlx-llm-server --model-path mlx-community/Mixtral-8x7B-v0.1-hf-4bit-mlx --adapter-file adapters.npz
```