Zero Shot Classification using multiGPU
Can we not use the Pipeline abstract class with multi GPU? I am trying do inference using pipeline(“zero-shot-classification”). It only utilises single GPU & throws CUDA out of memory memory error on 16GB 4x V100. How to go about using multiGPU(other than device=0)?
2023-08-14T23:35:18Z
I also have the same question.Is there a way to usepipelinefor inference with a “zero-shot-classification” model?I cannot use more than one GPU at a time withpipelinefor inference purposes.Thisconversation on GitHubis relevant but seems to point to the conclusion that it is not possible. However, if I am mistaken, please let me know.Toward the end of the conversation, it says that theacceleratemodule will help, but when I run the following:from transformers import pipeline 

pipe = pipeline(
                task="zero-shot-learning", 
                model="MoritzLaurer/DeBERTa-v3-base-mnli-fever-anli", 
                device_map="auto"
)The error that I get is:('Keyword argument not understood:', 'device_map')I am using the following GPU compute set up:GPU Info: 
  GPU 0: NVIDIA A10G
  GPU 1: NVIDIA A10G
  GPU 2: NVIDIA A10G
  GPU 3: NVIDIA A10G