Update README.md
Browse files
README.md
CHANGED
|
@@ -19,6 +19,17 @@ Original model: https://huggingface.co/Qwen/QVQ-72B-Preview
|
|
| 19 |
|
| 20 |
All quants made using imatrix option with dataset from [here](https://gist.github.com/bartowski1182/eb213dccb3571f863da82e99418f81e8)
|
| 21 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 22 |
## Prompt format
|
| 23 |
|
| 24 |
```
|
|
|
|
| 19 |
|
| 20 |
All quants made using imatrix option with dataset from [here](https://gist.github.com/bartowski1182/eb213dccb3571f863da82e99418f81e8)
|
| 21 |
|
| 22 |
+
## How to run
|
| 23 |
+
|
| 24 |
+
QwenVL models need to use the `llama-qwen2vl-cli` tool like so:
|
| 25 |
+
|
| 26 |
+
```
|
| 27 |
+
./llama-qwen2vl-cli -m /models/QVQ-72B-Preview-Q4_K_M.gguf \
|
| 28 |
+
--mmproj /models/mmproj-QVQ-72B-Preview-f16.gguf \
|
| 29 |
+
-p 'How many fingers does this hand have.' \
|
| 30 |
+
--image '/models/hand.jpg'
|
| 31 |
+
```
|
| 32 |
+
|
| 33 |
## Prompt format
|
| 34 |
|
| 35 |
```
|