Deepspeed script launcher vs accelerate script launcher for TRL
I’ve been trying to figure out the nature of the deepspeed integration, especially with respect to huggingface accelerate.It seems that the trainer uses accelerate to facilitate deepspeed. But when I look at the documentation, it seems that we still use deepspeed as the launcher, or the pytorch distributedeepspeed --num_gpus=2 your_program.py <normal cl args> --deepspeed ds_config.jsonorpython -m torch.distributed.launch --nproc_per_node=2 your_program.py <normal cl args>But it didn’t mention using the accelerate launcher. I’m confused by this since trainer uses accelerate to facilitate the deepspeed integration.As for the TRL library, it seems that it’s using the accelerate library for it’s trainers as well, but for the TRL library, the official way to launch it is to use the accelerate launcheraccelerate launch --config_file=examples/accelerate_configs/deepspeed_zero{1,2,3}.yaml --num_processes {NUM_GPUS} path_to_script.py --all_arguments_of_the_scripthuggingface.coExamplesWe’re on a journey to advance and democratize artificial intelligence through open source and open science.huggingface.coTraining customizationWe’re on a journey to advance and democratize artificial intelligence through open source and open science.I’m wondering if it’s still exchangeable with the deepspeed launcher, of it not, what’s the nature of the facilitation.