Getting GPU info from Accelerate
I was wondering if it were possible to print out which GPU a model is being trained on when using accelerate as well as how many GPUs will be used for training?
2022-07-06T19:26:06Z
aclifton314:GPU a model is being trained on when using accelerate as well as how many GPUs will be used for training?You should look at the output ofaccelerate envfrom the CLI. You can also configure this yourself by runningaccelerate configbefore training.Otherwise it’s highly dependent on how you start the script. E.g. if you calltorchrunitself and use accelerate in the script, it’ll use all of the GPUs available.How are you calling your script?
2022-07-06T19:49:18Z
@muellerzrThank you for your response!I call my scripttmp.pylikeaccelerate launch tmp.py
2022-07-06T19:53:09Z
aclifton314:I call my scripttmp.pylikeaccelerate launch tmp.pyThen it would be whateveraccelerate envhas configuredIf that hasn’t been configured yet then most likely its using all of your GPUs? (Though I thinkaccelerate launchwill give you an error if you haven’t configured it yet)
2022-07-06T19:57:14Z
I did runaccelerate configto set things up to use all 4 GPUs. Let me rephrase my question. I’d like to be able to log the total number of GPUs accelerate is using from within my python scripttmp.pyfor informational/debugging purposes. Is it possible to get that information from theAccelerateobject created in a python script?
2022-07-06T20:07:46Z
aclifton314:Let me rephrase my question. I’d like to be able to log the total number of GPUs accelerate is using from within my python scripttmp.pyfor informational/debugging purposes.Yes, for that you’d want the following information in Accelerator:Accelerator.num_processesandAccelerator.distributed_type.You can also gather these from theAcceleratorStateclass by doing:state = AcceleratorState()
num_devices, device_kind = state.num_processes, state.distributed_type(this does the same thing, you should probably just grab them from theacceleratorobject you have made)distributed_typewill return aDistributedType, which is an enum. You can dostr(device_kind)to get a string that looks like the following:'DistributedType.MULTI_GPU'
2022-07-06T20:27:19Z
That works like a charm!! Thank you!