

tokenizer = AutoTokenizer.from_pretrained("HuggingFaceH4/zephyr-7b-alpha")
model = AutoModelForCausalLM.from_pretrained(
     "HuggingFaceH4/zephyr-7b-alpha", device_map="auto", load_in_4bit=True
 )
set_seed(0)
prompt = """How many helicopters can a human eat in one sitting? Reply as a thug."""
model_inputs = tokenizer([prompt], return_tensors="pt").to("cuda")
input_length = model_inputs.input_ids.shape[1]
generated_ids = model.generate(**model_inputs, max_new_tokens=20)
print(tokenizer.batch_decode(generated_ids[:, input_length:], skip_special_tokens=True)[0])
"I'm not a thug, but i can tell you that a human cannot eat"
Oh no, it did not follow our instruction to reply as a thug! Let's see what happens when we write
a better prompt and use the right template for this model (through tokenizer.apply_chat_template)
set_seed(0)
messages = [
     {
         "role": "system",
         "content": "You are a friendly chatbot who always responds in the style of a thug",
     },
     {"role": "user", "content": "How many helicopters can a human eat in one sitting?"},
 ]
model_inputs = tokenizer.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt").to("cuda")
input_length = model_inputs.shape[1]
generated_ids = model.generate(model_inputs, do_sample=True, max_new_tokens=20)
print(tokenizer.batch_decode(generated_ids[:, input_length:], skip_special_tokens=True)[0])
'None, you thug. How bout you try to focus on more useful questions?'
As we can see, it followed a proper thug style 😎