Can a diffuser pipeline run on multiple GPUs?
Hey, we havethis sampleusingInstruct-pix2pixdiffuser . We observe that inference is faster on a multi-GPU instance than on a single-GPU instance ; is thepipe.to("cuda:" + gpu_id)running the pipeline on multiple GPUs? what explains the speedup on a multi-GPU machine vs single-GPU machine?
2023-05-15T10:41:59Z
The sample you shared is not using the Hugging Face DLC. We cannot help you with that.
2023-05-31T08:16:57Z
Hi@philschmid,I authored the code above to have an example handle the multi-gpu problem stated under:How to pass device_id to overriden functions? · Issue #66 · aws/sagemaker-huggingface-inference-toolkit · GitHub.Probably the huggingface-toolkit needs to be fixed (seeHow to pass device_id to overriden functions? · Issue #66 · aws/sagemaker-huggingface-inference-toolkit · GitHub) to work with multiple gpus in the same fashion.