llama-3.2-3B-Instruct

Sleeping

dntrplytch commited on Oct 4, 2024

Commit

34d32ce

verified ·

1 Parent(s): 9147ca6

update type

torch_dtype=torch.float16,

Files changed (1) hide show

app.py CHANGED Viewed

@@ -26,7 +26,7 @@ tokenizer = AutoTokenizer.from_pretrained(model_id)
 model = AutoModelForCausalLM.from_pretrained(
     model_id,
     device_map="auto",
-    torch_dtype=torch.bfloat16,
 )
 model.eval()

 model = AutoModelForCausalLM.from_pretrained(
     model_id,
     device_map="auto",
+    torch_dtype=torch.float16,
 )
 model.eval()