Checkpointing in each step
HiI use finetune_trainer.py, it saves the best model during checkpoints given an evaluation metric, so sometimes it calls the _save_checkpoint, but if the metric is not higher than the best saved one it would not save the cehckpoint in huggingface code, what I need is to write a callback, that save model+optimizer+scheduler to be called write after each time it calls the save_checkpoint to keep a copy of last updated model in the folder. I am not sure how to access optimizer/model/scheduler in the callbacks, I appreciate your input on this. thanks
2021-01-20T07:29:25Z
you could usesave_stpesarg to specify the number of update steps after which to save the checkpoint. If you use--evaluation_strategy steps
--eval_steps 50
--save_steps 50then it will eval and save checkpoint after every 50 steps