Training multiple times in one script
So I am trying to experiment with continual learning. As a part of that, I have to train a model several times. it will be a classification model using transformers.The easiest way is to save the model and load it each time, but I am using some cloud resources to store it, so it will be a real hassle to do that.So i want to train in 2-3 incremental stages in the same job that I run.So basically, after trainer.train() on one dataset, i want to continue training on a different dataset.For this, I am thinking of creating trainer subclass again, with the previous model and same previous parameters.My questionsis that the right way? is there a way without creating new trainer?which among trainer.modeland trainer.model_wrappedshould I use in the new trainer so that the trained parameters are retained?