Huggingface Optimizer
Hi, We should incorporate the newly released Sophia optimizer into huggingface !
2023-05-25T07:20:00Z
Hi@gordonhu! Optimizers are managed in Transformers so I recommend that you open a feature request there
2023-05-25T07:46:37Z
Thank you for letting me know!