# smol_llama-220M-GQA-32k-theta-sft-limarp

Experimental model meant to serve as a long-context speculative decoding model. This one is specifically for models trained on the LimaRP prompt format.

Created using [Doctor-Shotgun/smol_llama-220M-GQA-32k-theta-sft](https://huggingface.co/Doctor-Shotgun/smol_llama-220M-GQA-32k-theta-sft) and finetuning at 32768 context length on the LimaRP dataset.

This variant uses the rope theta (rope frequency base) method for context extension.

The trained instruction format is LimaRP Alpaca:
```
### Instruction:
Character's Persona: {bot character description}

User's Persona: {user character description}

Scenario: {what happens in the story}

Play the role of Character. Taking the above information into consideration, you must engage in a roleplaying chat with User below this line. Do not write dialogues and narration for User.

### Input:
User: {utterance}

### Response:
Character: {utterance}

### Input
User: {utterance}

### Response:
Character: {utterance}

(etc.)
```
{"language": ["en"], "tags": ["llama", "llama 2", "smol_llama"], "datasets": ["lemonilia/LimaRP"]}
transformers pytorch llama text-generation llama 2 smol_llama en dataset:lemonilia/LimaRP autotrain_compatible endpoints_compatible text-generation-inference region:us