Bigger batch size, the lower throughput and GPU usage？
I’m trainingbert-baseon single node with 8xA100 GPU, usingrun_mlm.pyscript.when the batch size set to256, the throughput is 8000 sample/s, and GPU usage is 80%.when the batch size set to384, the throughput is 4000 sample/s, and GPU usage is 50%.What is the reason for this phenomenon？Data IO became bottleneck?
2022-07-16T00:10:03Z
Hi, I’m having a similar issue with ViT but I am getting very inconsistent and low GPU usage. Did you figure this out? I think it is IO as well