# Felladrin/TinyMistral-248M-SFT-v3-GGUF

Quantized GGUF model files for [TinyMistral-248M-SFT-v3](https://huggingface.co/Felladrin/TinyMistral-248M-SFT-v3) from [Felladrin](https://huggingface.co/Felladrin)


| Name | Quant method | Size |
| ---- | ---- | ---- |
| [tinymistral-248m-sft-v3.fp16.gguf](https://huggingface.co/afrideva/TinyMistral-248M-SFT-v3-GGUF/resolve/main/tinymistral-248m-sft-v3.fp16.gguf) | fp16 | 497.75 MB  |
| [tinymistral-248m-sft-v3.q2_k.gguf](https://huggingface.co/afrideva/TinyMistral-248M-SFT-v3-GGUF/resolve/main/tinymistral-248m-sft-v3.q2_k.gguf) | q2_k | 116.20 MB  |
| [tinymistral-248m-sft-v3.q3_k_m.gguf](https://huggingface.co/afrideva/TinyMistral-248M-SFT-v3-GGUF/resolve/main/tinymistral-248m-sft-v3.q3_k_m.gguf) | q3_k_m | 131.01 MB  |
| [tinymistral-248m-sft-v3.q4_k_m.gguf](https://huggingface.co/afrideva/TinyMistral-248M-SFT-v3-GGUF/resolve/main/tinymistral-248m-sft-v3.q4_k_m.gguf) | q4_k_m | 156.60 MB  |
| [tinymistral-248m-sft-v3.q5_k_m.gguf](https://huggingface.co/afrideva/TinyMistral-248M-SFT-v3-GGUF/resolve/main/tinymistral-248m-sft-v3.q5_k_m.gguf) | q5_k_m | 180.16 MB  |
| [tinymistral-248m-sft-v3.q6_k.gguf](https://huggingface.co/afrideva/TinyMistral-248M-SFT-v3-GGUF/resolve/main/tinymistral-248m-sft-v3.q6_k.gguf) | q6_k | 205.20 MB  |
| [tinymistral-248m-sft-v3.q8_0.gguf](https://huggingface.co/afrideva/TinyMistral-248M-SFT-v3-GGUF/resolve/main/tinymistral-248m-sft-v3.q8_0.gguf) | q8_0 | 265.26 MB  |



## Original Model Card:
# Locutusque's TinyMistral-248M trained on OpenAssistant TOP-1 Conversation Threads

- Base model: [Locutusque/TinyMistral-248M](https://huggingface.co/Locutusque/TinyMistral-248M/blob/90b89d18fdf27937dc04ab8a9b543c5af2991c7f/README.md)
- Dataset: [OpenAssistant/oasst_top1_2023-08-25](https://huggingface.co/datasets/OpenAssistant/oasst_top1_2023-08-25)

## Recommended Prompt Format

```
<|im_start|>user
{message}<|im_end|>
<|im_start|>assistant
```

## How it was trained

```ipython
%pip install autotrain-advanced

!autotrain setup

!autotrain llm \
    --train \
    --trainer "sft" \
    --model './TinyMistral-248M/' \
    --model_max_length 4096 \
    --block-size 1024 \
    --project-name 'trained-model' \
    --data-path "OpenAssistant/oasst_top1_2023-08-25" \
    --train_split "train" \
    --valid_split "test" \
    --text-column "text" \
    --lr 1e-5 \
    --train_batch_size 2 \
    --epochs 5 \
    --evaluation_strategy "steps" \
    --save-strategy "steps" \
    --save-total-limit 2 \
    --warmup-ratio 0.05 \
    --weight-decay 0.0 \
    --gradient-accumulation 8 \
    --logging-steps 10 \
    --scheduler "constant"
```
{"license": "apache-2.0", "tags": ["autotrain", "text-generation", "gguf", "ggml", "quantized", "q2_k", "q3_k_m", "q4_k_m", "q5_k_m", "q6_k", "q8_0"], "datasets": ["OpenAssistant/oasst_top1_2023-08-25"], "model_name": "TinyMistral-248M-SFT-v3", "base_model": "Felladrin/TinyMistral-248M-SFT-v3", "inference": false, "model_creator": "Felladrin", "pipeline_tag": "text-generation", "quantized_by": "afrideva", "widget": [{"text": "<|im_start|>user\nWrite the specs of a game about trolls and warriors in a fantasy world.<|im_end|>\n<|im_start|>assistant\nThe game is an adventure game that takes place on a planet, where players must explore their unique abilities to survive. Players can use different strategies such as collecting items or trading them for gold or silver coins, but they also need to learn how to deal with obstacles and find new ways to escape.<|im_end|>\n<|im_start|>user\nCould you tell me something curious about the Earth?<|im_end|>\n<|im_start|>assistant\nThe planet is a large, rocky world with an atmosphere of 10 billion years old and a surface area around 25 million miles (36 million kilometers) wide.<|im_end|>\n<|im_start|>user\nWhat are some potential applications for quantum computing?<|im_end|>\n<|im_start|>assistant"}]}
gguf autotrain text-generation ggml quantized q2_k q3_k_m q4_k_m q5_k_m q6_k q8_0 dataset:OpenAssistant/oasst_top1_2023-08-25 base_model:Felladrin/TinyMistral-248M-SFT-v3 license:apache-2.0 region:us