Resume_from_checkpoint does not configure learning rate scheduler correctly
I use the code(https://github.com/shibing624/MedicalGPT/blob/main/pretraining.py) to do pretraining. When I use resume_from_checkpoint to restart the training process from the checkpoint of 2000 step, the learning rate scheduler is as follows.截屏2023-07-31 13.12.443200×872 206 KB
2023-09-14T05:43:35Z
Same problem! Any progress here?
2023-11-25T14:45:00Z
Same problem@staswould you like to take a look?
2023-11-28T05:24:44Z
You will need to file an issue with whoever wrote this code as I’m not familiar with this repo.Also I no longer maintain the HF transformers/Deepspeed integration if it’s related to HF Transformers.Currently all HF transformers/Deepspeed integration is done via HF Accelerate.