README.md

---
license: mit
---

# Miquliz-120b-v2.0-FP8-dynamic
![image/jpeg](/static-proxy?url=https%3A%2F%2Fcdn-uploads.huggingface.co%2Fproduction%2Fuploads%2F6303ca537373aacccd85d8a7%2FvmCAhJCpF0dITtCVxlYET.jpeg%3C%2Fspan%3E)%3C!-- HTML_TAG_END -->

This quant was made for [infermatic.ai](https://infermatic.ai/)

Dynamic FP8 quant of [Miquliz 120B v2.0](https://huggingface.co/wolfram/miquliz-120b-v2.0) made with AutoFP8.

## Model Details

- Max Context: 32768 tokens
- Layers: 140

### Prompt template: Mistral

```
<s>[INST] {prompt} [/INST]
```