Any good code/tutorial that is shows how to do inference with Llama 2 70b on multiple GPUs with accelerate?
Do you know of any good code/tutorial that is shows how to do inference with Llama 2 70b on multiple GPUs with accelerate?
2023-11-27T18:00:42Z
Hey@vbachi, you can check this doc:Handling big models for inference