Upload README.md
Browse files
README.md
CHANGED
|
@@ -12,6 +12,13 @@ tags:
|
|
| 12 |
## mradermacher's superb gguf version, thank you for your conscientious and responsible dedication.
|
| 13 |
- https://huggingface.co/mradermacher/Phi-3-song-lyrics-1.0-i1-GGUF
|
| 14 |
- https://huggingface.co/mradermacher/Phi-3-song-lyrics-1.0-GGUF
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 15 |
|
| 16 |
|
| 17 |
|
|
|
|
| 12 |
## mradermacher's superb gguf version, thank you for your conscientious and responsible dedication.
|
| 13 |
- https://huggingface.co/mradermacher/Phi-3-song-lyrics-1.0-i1-GGUF
|
| 14 |
- https://huggingface.co/mradermacher/Phi-3-song-lyrics-1.0-GGUF
|
| 15 |
+
## These are my own quantizations (updated almost daily).
|
| 16 |
+
The difference with normal quantizations is that I quantize the output and embed tensors to f16.
|
| 17 |
+
and the other tensors to 15_k,q6_k or q8_0.
|
| 18 |
+
This creates models that are little or not degraded at all and have a smaller size.
|
| 19 |
+
They run at about 3-6 t/sec on CPU only using llama.cpp
|
| 20 |
+
And obviously faster on computers with potent GPUs
|
| 21 |
+
- the fast cat at [ZeroWw/Phi-3-song-lyrics-1.0-GGUF](https://huggingface.co/ZeroWw/Phi-3-song-lyrics-1.0-GGUF)
|
| 22 |
|
| 23 |
|
| 24 |
|