Is it possible to see what batch size is being used in deepspeed training with auto batch size?
Is it possible to see what batch size is being used in deepspeed training with auto batch size?
2023-07-14T23:46:03Z
48%|██████████████████████████████████▏                                    | 13/27 [06:48<09:26, 40.49s/it]To test, I’m training 1 epoch. 1 epoch in happening in 27 steps. There are 1700 training examples, so it is 63 examples per step. Does this mean that the batch size is 1700/27 ~ 63, and the micro batch size is 63/8 ~= 8  (there are 8 GPUs)? I believe “batch” is defined as the number of training examples involved with a gradient accumulation and weight update, while micro batch is number of examples being processed by a single GPU at once.