Update README.md
Browse files
README.md
CHANGED
|
@@ -26,6 +26,37 @@ All quants were made using the imatrix option and Bartowski's [calibration file]
|
|
| 26 |
|
| 27 |
# Perplexity table (the lower the better)
|
| 28 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 29 |
<hr>
|
| 30 |
|
| 31 |
# Qwen2.5-3B-Instruct
|
|
|
|
| 26 |
|
| 27 |
# Perplexity table (the lower the better)
|
| 28 |
|
| 29 |
+
| Quant | Size (MB) | PPL | Size (%) | Accuracy (%) | PPL error rate |
|
| 30 |
+
| ------- | --------- | -------- | -------- | ------------ | -------------- |
|
| 31 |
+
| IQ1_S | 755 | 112.0612 | 12.81 | 8.02 | 0.97138 |
|
| 32 |
+
| IQ1_M | 811 | 42.7456 | 13.76 | 21.03 | 0.34718 |
|
| 33 |
+
| IQ2_XXS | 905 | 25.2117 | 15.36 | 35.65 | 0.20222 |
|
| 34 |
+
| IQ2_XS | 984 | 15.9149 | 16.7 | 56.48 | 0.11965 |
|
| 35 |
+
| IQ2_S | 1013 | 14.5975 | 17.19 | 61.58 | 0.1082 |
|
| 36 |
+
| IQ2_M | 1088 | 12.8779 | 18.46 | 69.8 | 0.09436 |
|
| 37 |
+
| Q2_K_S | 1143 | 13.0878 | 19.4 | 68.68 | 0.09636 |
|
| 38 |
+
| Q2_K | 1216 | 11.8001 | 20.63 | 76.18 | 0.08674 |
|
| 39 |
+
| IQ3_XXS | 1224 | 10.6049 | 20.77 | 84.76 | 0.07572 |
|
| 40 |
+
| IQ3_XS | 1328 | 10.0306 | 22.54 | 89.61 | 0.06975 |
|
| 41 |
+
| Q3_K_S | 1387 | 15.5457 | 23.54 | 57.82 | 0.11941 |
|
| 42 |
+
| IQ3_S | 1390 | 9.9591 | 23.59 | 90.26 | 0.06984 |
|
| 43 |
+
| IQ3_M | 1420 | 9.9957 | 24.1 | 89.93 | 0.06962 |
|
| 44 |
+
| Q3_K_M | 1517 | 14.0989 | 25.74 | 63.76 | 0.10568 |
|
| 45 |
+
| Q3_K_L | 1629 | 13.8579 | 27.64 | 64.86 | 0.10372 |
|
| 46 |
+
| IQ4_XS | 1659 | 9.2935 | 28.15 | 96.72 | 0.06517 |
|
| 47 |
+
| IQ4_NL | 1741 | 9.2824 | 29.54 | 96.84 | 0.06503 |
|
| 48 |
+
| Q4_0 | 1744 | 9.485 | 29.59 | 94.77 | 0.06626 |
|
| 49 |
+
| Q4_K_S | 1750 | 9.2573 | 29.7 | 97.1 | 0.06485 |
|
| 50 |
+
| Q4_K_M | 1841 | 9.2305 | 31.24 | 97.38 | 0.06475 |
|
| 51 |
+
| Q4_1 | 1904 | 9.2746 | 32.31 | 96.92 | 0.06512 |
|
| 52 |
+
| Q5_K_S | 2070 | 9.1338 | 35.13 | 98.41 | 0.06402 |
|
| 53 |
+
| Q5_0 | 2075 | 9.1513 | 35.21 | 98.22 | 0.06413 |
|
| 54 |
+
| Q5_K_M | 2122 | 9.1339 | 36.01 | 98.41 | 0.06407 |
|
| 55 |
+
| Q5_1 | 2235 | 9.1231 | 37.93 | 98.53 | 0.06386 |
|
| 56 |
+
| Q6_K | 2421 | 9.069 | 41.08 | 99.12 | 0.06342 |
|
| 57 |
+
| Q8_0 | 3134 | 9.0114 | 53.18 | 99.75 | 0.06285 |
|
| 58 |
+
| F16 | 5893 | 8.9888 | 100 | 100 | 0.06268 |
|
| 59 |
+
|
| 60 |
<hr>
|
| 61 |
|
| 62 |
# Qwen2.5-3B-Instruct
|