# Felladrin/Llama-160M-Chat-v1-GGUF

Quantized GGUF model files for [Llama-160M-Chat-v1](https://huggingface.co/Felladrin/Llama-160M-Chat-v1) from [Felladrin](https://huggingface.co/Felladrin)


| Name | Quant method | Size |
| ---- | ---- | ---- |
| [llama-160m-chat-v1.fp16.gguf](https://huggingface.co/afrideva/Llama-160M-Chat-v1-GGUF/resolve/main/llama-160m-chat-v1.fp16.gguf) | fp16 | 326.58 MB  |
| [llama-160m-chat-v1.q2_k.gguf](https://huggingface.co/afrideva/Llama-160M-Chat-v1-GGUF/resolve/main/llama-160m-chat-v1.q2_k.gguf) | q2_k | 77.23 MB  |
| [llama-160m-chat-v1.q3_k_m.gguf](https://huggingface.co/afrideva/Llama-160M-Chat-v1-GGUF/resolve/main/llama-160m-chat-v1.q3_k_m.gguf) | q3_k_m | 87.54 MB  |
| [llama-160m-chat-v1.q4_k_m.gguf](https://huggingface.co/afrideva/Llama-160M-Chat-v1-GGUF/resolve/main/llama-160m-chat-v1.q4_k_m.gguf) | q4_k_m | 104.03 MB  |
| [llama-160m-chat-v1.q5_k_m.gguf](https://huggingface.co/afrideva/Llama-160M-Chat-v1-GGUF/resolve/main/llama-160m-chat-v1.q5_k_m.gguf) | q5_k_m | 119.04 MB  |
| [llama-160m-chat-v1.q6_k.gguf](https://huggingface.co/afrideva/Llama-160M-Chat-v1-GGUF/resolve/main/llama-160m-chat-v1.q6_k.gguf) | q6_k | 135.00 MB  |
| [llama-160m-chat-v1.q8_0.gguf](https://huggingface.co/afrideva/Llama-160M-Chat-v1-GGUF/resolve/main/llama-160m-chat-v1.q8_0.gguf) | q8_0 | 174.33 MB  |



## Original Model Card:
# A Llama Chat Model of 160M Parameters

- Base model: [JackFram/llama-160m](https://huggingface.co/JackFram/llama-160m)
- Datasets:
  - [ehartford/wizard_vicuna_70k_unfiltered](https://huggingface.co/datasets/ehartford/wizard_vicuna_70k_unfiltered)
  - [totally-not-an-llm/EverythingLM-data-V3](https://huggingface.co/datasets/totally-not-an-llm/EverythingLM-data-V3)
  - [Open-Orca/SlimOrca-Dedup](https://huggingface.co/datasets/Open-Orca/SlimOrca-Dedup)
  - [databricks/databricks-dolly-15k](https://huggingface.co/datasets/databricks/databricks-dolly-15k)
  - [THUDM/webglm-qa](https://huggingface.co/datasets/THUDM/webglm-qa)
- Availability in other ML formats:
  - GGUF: [Felladrin/gguf-Llama-160M-Chat-v1](https://huggingface.co/Felladrin/gguf-Llama-160M-Chat-v1)
  - ONNX: [Felladrin/onnx-Llama-160M-Chat-v1](https://huggingface.co/Felladrin/onnx-Llama-160M-Chat-v1)

## Recommended Prompt Format

The recommended prompt format is as follows:

```
<|im_start|>system
{system_message}<|im_end|>
<|im_start|>user
{user_message}<|im_end|>
<|im_start|>assistant
```

## Recommended Inference Parameters

To get the best results, prefer using [contrastive search](https://huggingface.co/docs/transformers/main/en/generation_strategies#contrastive-search) for inference:

```yml
penalty_alpha: 0.5
top_k: 5
```
{"license": "other", "tags": ["text-generation", "gguf", "ggml", "quantized", "q2_k", "q3_k_m", "q4_k_m", "q5_k_m", "q6_k", "q8_0"], "datasets": ["ehartford/wizard_vicuna_70k_unfiltered", "totally-not-an-llm/EverythingLM-data-V3", "Open-Orca/SlimOrca-Dedup", "databricks/databricks-dolly-15k", "THUDM/webglm-qa"], "model_name": "Llama-160M-Chat-v1", "base_model": "Felladrin/Llama-160M-Chat-v1", "inference": false, "model_creator": "Felladrin", "pipeline_tag": "text-generation", "quantized_by": "afrideva", "widget": [{"text": "<|im_start|>system\nYou are a helpful assistant, who answers with empathy.<|im_end|>\n<|im_start|>user\nGot a question for you!<|im_end|>\n<|im_start|>assistant\nSure! What's it?<|im_end|>\n<|im_start|>user\nWhy do you love cats so much!? \ud83d\udc08<|im_end|>\n<|im_start|>assistant"}, {"text": "<|im_start|>system\nYou are a helpful assistant who answers user's questions with empathy.<|im_end|>\n<|im_start|>user\nWho is Mona Lisa?<|im_end|>\n<|im_start|>assistant"}, {"text": "<|im_start|>system\nYou are a helpful assistant who provides concise responses.<|im_end|>\n<|im_start|>user\nHeya!<|im_end|>\n<|im_start|>assistant\nHi! How may I help you today?<|im_end|>\n<|im_start|>user\nI need to build a simple website. Where should I start learning about web development?<|im_end|>\n<|im_start|>assistant"}, {"text": "<|im_start|>user\nInvited some friends to come home today. Give me some ideas for games to play with them!<|im_end|>\n<|im_start|>assistant"}, {"text": "<|im_start|>system\nYou are a helpful assistant who answers user's questions with details and curiosity.<|im_end|>\n<|im_start|>user\nWhat are some potential applications for quantum computing?<|im_end|>\n<|im_start|>assistant"}, {"text": "<|im_start|>system\nYou are a helpful assistant who gives creative responses.<|im_end|>\n<|im_start|>user\nWrite the specs of a game about mages in a fantasy world.<|im_end|>\n<|im_start|>assistant"}, {"text": "<|im_start|>system\nYou are a helpful assistant who answers user's questions with details.<|im_end|>\n<|im_start|>user\nTell me about the pros and cons of social media.<|im_end|>\n<|im_start|>assistant"}, {"text": "<|im_start|>system\nYou are a helpful assistant who answers user's questions with confidence.<|im_end|>\n<|im_start|>user\nWhat is a dog?<|im_end|>\n<|im_start|>assistant\nA dog is a four-legged, domesticated animal that is a member of the class Mammalia, which includes all mammals. Dogs are known for their loyalty, playfulness, and ability to be trained for various tasks. They are also used for hunting, herding, and as service animals.<|im_end|>\n<|im_start|>user\nWhat is the color of an apple?<|im_end|>\n<|im_start|>assistant"}]}
gguf text-generation ggml quantized q2_k q3_k_m q4_k_m q5_k_m q6_k q8_0 dataset:ehartford/wizard_vicuna_70k_unfiltered dataset:totally-not-an-llm/EverythingLM-data-V3 dataset:Open-Orca/SlimOrca-Dedup dataset:databricks/databricks-dolly-15k dataset:THUDM/webglm-qa base_model:Felladrin/Llama-160M-Chat-v1 license:other region:us