Using gradient_accumulation_steps does not give the same results
I am using the summarizationexamplethis is my script for traing using 3 gpusaccelerate launch --config_file='./accelerate.yaml' run_summarization_no_trainer_test.py  --seed=42 --preprocessing_num_workers=1 --weight_decay='0.001' --output_dir="samsum_exp/t5/base_3" --per_device_train_batch_size=1 --per_device_eval_batch_size=1    --num_train_epochs=10  --model_name_or_path='t5-base' --num_beams=5 --with_tracking --report_to='wandb' --checkpointing_steps='epoch'  --dataset_name="samsum" --source_prefix "summarize: " --gradient_accumulation_steps=2using 6 gpusaccelerate launch --config_file='./accelerate.yaml' run_summarization_no_trainer_test.py  --seed=42 --preprocessing_num_workers=1 --weight_decay='0.001' --output_dir="samsum_exp/t5/base_6" --per_device_train_batch_size=1 --per_device_eval_batch_size=1    --num_train_epochs=10  --model_name_or_path='t5-base' --num_beams=5 --with_tracking --report_to='wandb' --checkpointing_steps='epoch'  --dataset_name="samsum" --source_prefix "summarize: " --gradient_accumulation_steps=1The resuled metrics are as in figure below:image969×402 32.6 KBAccording to thisansweIt supposed to get identical result but it is not.