
# StableLM-Tuned-Alpha 16-bit

## Model Description

16-bit version of `StableLM-Tuned-Alpha` compressed for the sake of speed and memory usage. No other changes were made. Original model: https://huggingface.co/stabilityai/stablelm-tuned-alpha-7b

## Usage

Get started chatting with `StableLM-Tuned-Alpha 16-bit` by using the following code snippet:

```python
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer, StoppingCriteria, StoppingCriteriaList
tokenizer = AutoTokenizer.from_pretrained("vvsotnikov/stablelm-tuned-alpha-7b-16bit")
model = AutoModelForCausalLM.from_pretrained("vvsotnikov/stablelm-tuned-alpha-7b-16bit", torch_dtype=torch.float16)
model.cuda()
class StopOnTokens(StoppingCriteria):
    def __call__(self, input_ids: torch.LongTensor, scores: torch.FloatTensor, **kwargs) -> bool:
        stop_ids = [50278, 50279, 50277, 1, 0]
        for stop_id in stop_ids:
            if input_ids[0][-1] == stop_id:
                return True
        return False
system_prompt = """<|SYSTEM|># StableLM Tuned (Alpha version)
- StableLM is a helpful and harmless open-source AI language model developed by StabilityAI.
- StableLM is excited to be able to help the user, but will refuse to do anything that could be considered harmful to the user.
- StableLM is more than just an information source, StableLM is also able to write poetry, short stories, and make jokes.
- StableLM will refuse to participate in anything that could harm a human.
"""
prompt = f"{system_prompt}<|USER|>What's your mood today?<|ASSISTANT|>"
inputs = tokenizer(prompt, return_tensors="pt").to("cuda")
tokens = model.generate(
  **inputs,
  max_new_tokens=64,
  temperature=0.7,
  do_sample=True,
  stopping_criteria=StoppingCriteriaList([StopOnTokens()])
)
print(tokenizer.decode(tokens[0], skip_special_tokens=True))
```
{"language": ["en"], "license": "cc-by-nc-sa-4.0", "tags": ["causal-lm"], "datasets": ["dmayhem93/ChatCombined", "tatsu-lab/alpaca", "nomic-ai/gpt4all_prompt_generations", "Dahoas/full-hh-rlhf", "jeffwan/sharegpt_vicuna", "HuggingFaceH4/databricks_dolly_15k"]}
transformers pytorch gpt_neox text-generation causal-lm en dataset:dmayhem93/ChatCombined dataset:tatsu-lab/alpaca dataset:nomic-ai/gpt4all_prompt_generations dataset:Dahoas/full-hh-rlhf dataset:jeffwan/sharegpt_vicuna dataset:HuggingFaceH4/databricks_dolly_15k license:cc-by-nc-sa-4.0 autotrain_compatible endpoints_compatible text-generation-inference region:us
TAGS
#transformers #pytorch #gpt_neox #text-generation #causal-lm #en #dataset-dmayhem93/ChatCombined #dataset-tatsu-lab/alpaca #dataset-nomic-ai/gpt4all_prompt_generations #dataset-Dahoas/full-hh-rlhf #dataset-jeffwan/sharegpt_vicuna #dataset-HuggingFaceH4/databricks_dolly_15k #license-cc-by-nc-sa-4.0 #autotrain_compatible #endpoints_compatible #text-generation-inference #region-us 


# StableLM-Tuned-Alpha 16-bit

## Model Description

16-bit version of 'StableLM-Tuned-Alpha' compressed for the sake of speed and memory usage. No other changes were made. Original model: URL

## Usage

Get started chatting with 'StableLM-Tuned-Alpha 16-bit' by using the following code snippet:


# StableLM-Tuned-Alpha 16-bit ## Model Description

16-bit version of 'StableLM-Tuned-Alpha' compressed for the sake of speed and memory usage. No other changes were made. Original model: URL ## Usage

Get started chatting with 'StableLM-Tuned-Alpha 16-bit' by using the following code snippet:
TAGS
#transformers #pytorch #gpt_neox #text-generation #causal-lm #en #dataset-dmayhem93/ChatCombined #dataset-tatsu-lab/alpaca #dataset-nomic-ai/gpt4all_prompt_generations #dataset-Dahoas/full-hh-rlhf #dataset-jeffwan/sharegpt_vicuna #dataset-HuggingFaceH4/databricks_dolly_15k #license-cc-by-nc-sa-4.0 #autotrain_compatible #endpoints_compatible #text-generation-inference #region-us 
 # StableLM-Tuned-Alpha 16-bit ## Model Description

16-bit version of 'StableLM-Tuned-Alpha' compressed for the sake of speed and memory usage. No other changes were made. Original model: URL ## Usage

Get started chatting with 'StableLM-Tuned-Alpha 16-bit' by using the following code snippet:
TAGS
#transformers #pytorch #gpt_neox #text-generation #causal-lm #en #dataset-dmayhem93/ChatCombined #dataset-tatsu-lab/alpaca #dataset-nomic-ai/gpt4all_prompt_generations #dataset-Dahoas/full-hh-rlhf #dataset-jeffwan/sharegpt_vicuna #dataset-HuggingFaceH4/databricks_dolly_15k #license-cc-by-nc-sa-4.0 #autotrain_compatible #endpoints_compatible #text-generation-inference #region-us 
# StableLM-Tuned-Alpha 16-bit## Model Description

16-bit version of 'StableLM-Tuned-Alpha' compressed for the sake of speed and memory usage. No other changes were made. Original model: URL## Usage

Get started chatting with 'StableLM-Tuned-Alpha 16-bit' by using the following code snippet: