How to use specific gpu in accelerate?
I want to use GPUs with different conditions.But I think the accelerator.device() is always cuda:0. even I wanted to rewrite it like cuda:1 or cuda:2 but it couldn’t be modified. How to fix this problem?
2023-11-20T15:02:24Z
You can specify eitherCUDA_VISIBLE_DEVICESor use the--gpu_idsparam to either your config oraccelerate launch
2023-11-21T05:27:13Z
I’m a bit confused by your response. In fact, I already added the following code at the beginning of my file, specifying GPU number 6:os.environ['CUDA_VISIBLE_DEVICES'] = str(6)
device = torch.device(f"cuda:{args.gpu}" if torch.cuda.is_available() else "cpu")However, after usingaccelerator = Accelerator(), when I checkaccelerator.device, it still showsdevice(type='cuda')and remains on device 0. If I manually setaccelerator.device = self.device, it throws an error:AttributeError: can't set attribute. Could you please provide more detailed instructions on how to specify the GPU using the Accelerator? I would greatly appreciate it.
2023-11-21T05:57:38Z
exactly the same situation!!
2023-11-21T13:56:26Z
hanjohn:os.environ['CUDA_VISIBLE_DEVICES'] = str(6)You cannot do this in your python file like that, this has to be donebeforeyour python file has been called, or before torch/accelerate/anything that init’s the GPU has been imported (possibly).So solutions:accelerate launch --gpu_ids 6 myscript.pyCUDA_VISIBLE_DEVICES=6 python myscript.py(Do not know if this last solution will actually work, I haven’t tried it before)import os
os.environ["CUDA_VISIBLE_DEVICES"] = str(6)
import torch
...
2023-11-21T14:59:58Z
Thank you for your prompt reply. However, when following your guidance I encountered the following error:RuntimeError: CUDA error: invalid device ordinalCUDA kernel errors might be asynchronously reported at some other API call,so the stacktrace below might be incorrect.
2023-11-22T03:09:15Z
Thank you I solved it
2023-11-22T04:57:16Z
Hi, Could you please explain in more detail how you solved this problem? I’m interested to understand the steps you took to resolve it.
2023-11-22T05:14:16Z
When changing your GPU through os.environ[‘CUDA_VISIBLE_DEVICES’], it is important that the env code should come first before all import sections. I think I failed when I tried to change GPU after all the other Accelerate and torch or tensorflow modules had been imported, suggesting that the problem of the ordinal environment codes order is the biggest.
2023-11-23T00:49:05Z
Thank you I solved it
2024-04-25T19:33:41Z
None of this worked for me. Everywhere I added CUDA_VISIBLE_DEVICES=4 but somehow it is taking 0. Nowhere in the code have I specified 0. This is the crappiest thing I’ve ever experienced. Wasted 1 hour and still not resolved. Kindly change the api to something simpler and respect the environment variables. Here is the code I’m using:GitHub - thuanz123/realfill