Which method is use HF Trainer with multiple GPU?
According to the following question, the trainer will handle multiple GPU work. What is the method it uses?DataParallel (DP) or TensorParallel (TP) or PipelineParallel (PP) or DPP, what?Training using multiple GPUsBeginnersI would like to train some models to multiple GPUs. 
Let suppose that I use model from HF library, but I am  using my own trainers,dataloader,collators etc. 
Where I should focus to implement multiple GPU training? I need to make changes only in the Trainer class? If yes, can you give me a brief description? 
Thank you in avance.
2022-12-02T15:04:12Z
Based onthis line of codeit looks like it is usingnn.DataParallel, however I haven’t fully looked at every line of theTrainerclass so they may also be using other methods in other points.
2022-12-03T01:56:42Z
Thank you for your information. Old Trainer documents have to configure that. But new document doese not mention it. Old Doc -Trainer — transformers 4.7.0 documentation
2023-06-19T04:56:01Z
hi@AndreaSottana, sorry I am trying to fine tune got-neo because of the Cuda memory issue I need to use multiple GPU. I use the trainer in hugging face which I understand it will use multiple GPu . but my results are very strange and very different than when I use 1 GPU. Would you please help me how you use multiple GPU for fine tunning the model.?Many thanks
2023-06-19T04:56:42Z
Hi@Indramalsorry I am trying to fine tune got-neo because of the Cuda memory issue I need to use multiple GPU. I use the trainer in hugging face which I understand it will use multiple GPu . but my results are very strange and very different than when I use 1 GPU. Would you please help me how you use multiple GPU for fine tunning the model.?Many thanks