Trainer.predict in parallel not supported!
Currently doing any inference via trainer.predict only uses 1 gpu to do all the computations. Has someone done any parallelization for this ? Split the data among all available gpus and do inference, aggregate all metrics once all processes are done ?
2022-11-02T13:56:20Z
It is supported and you can have a look at ourexampleswhich all use distributed evaluation if you have several GPUs.
2022-11-02T16:49:33Z
No, it doesn’t run it parallel. Trainer.predict() keeps using 1 gpu and rest of 7 sit idle.Different from that, I made my own which divides data and trainer.predicts on all N gpus and then gathers output back. This reduces time N times vs default usage of trainer.predict()