| library_name: transformers | |
| license: other | |
| license_name: llama3.3 | |
| base_model: TheDrummer/Anubis-70B-v1 | |
| tags: | |
| - exl2 | |
| # Anubis-70B-v1 - EXL2 2.25bpw | |
| %3C!-- HTML_TAG_END --> | |
| This is a 2.25bpw EXL2 quant of [TheDrummer/Anubis-70B-v1](https://huggingface.co/TheDrummer/Anubis-70B-v1) | |
| Details about the model can be found at the above model page. | |
| ## EXL2 Version | |
| These quants were made with exllamav2 version 0.2.4. Quants made on this version of EXL2 may not work on older versions of the exllamav2 library. | |
| If you have problems loading these models, please update Text Generation WebUI to the latest version. | |
| ## Perplexity Scoring | |
| Below are the perplexity scores for the EXL2 models. A lower score is better. | |
| | Quant Level | Perplexity Score | | |
| |-------------|------------------| | |
| | 5.0 | 4.5616 | | |
| | 4.5 | 4.6373 | | |
| | 4.0 | 4.7632 | | |
| | 3.5 | 5.1384 | | |
| | 3.0 | 7.3087 | | |
| | 2.75 | 7.8539 | | |
| | 2.5 | 7.8219 | | |
| | 2.25 | 7.6464 | | |