Why is `accelerator.save` saving once for each node?
Probably a dumb question: why isaccelerator.saveonce for each node instead of once for all nodes? i.e. conditionedlocal_process_index==0instead ofprocess_index==0seeaccelerate/other.py at cb54e1023e17763fa7f11d1de1307f85ba8b6c31 · huggingface/accelerate · GitHub
2022-08-30T04:05:00Z
Hello, I think it is done so that the user can load the model on any node later on.@sguggerand@muellerzrmight have more to add on this.
2022-08-31T20:12:54Z
Exactly. We could add an option for multinode systems that share the same file storage though.