How do you know which parameter is used for ZeRO?
Quoting fromhereIf you’re migrating from ZeRO-2 configuration note thatallgather_partitions,allgather_bucket_sizeandreduce_scatterconfiguration parameters are not used in ZeRO-3. If you keep these in the config file they will just be ignored.How did you know? Because when I read documentation of deepspeed configuration. There is never any mention about which parameters will be ignored under ZeRO-3.