Hugging Face Llama-2 (7b) taking too much time while inferencing
Hello everyone, I am trying to use Llama-2 (7b) from Hugging face. With below code I was able to load the model successfully but when I am trying to generate the output its taking forever.Codefrom transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("Llama-2-7b-hf")
model = AutoModelForCausalLM.from_pretrained("Llama-2-7b-hf")

input_ids = tokenizer.encode("What is LLM?", return_tensors="pt")

output = model.generate(
        input_ids,
        temperature=0,
        max_new_tokens=100
    )

generated_text = tokenizer.decode(output[0])
print(generated_text)Model files downloaded fromLlama-2-7b-hfHardware: Macbook Pro (M2 Pro) 16 GB RAM
2024-06-23T10:25:43Z
Hi, did you find a solution?