Same loss on multiple nodes
Hi FriendsI’m training a model on two nodes (using Huggingface Trainer) and in the console, both nodes log the same loss. Is that expected or are both nodes seeing the same training data?
2023-11-02T11:09:04Z
Yes, that is how DDP works