The best way to load pertained weights then continue training BERT?
What is the best way to load pertained weights then continue training BERT using my server’s GPU? Should I load the model as usual then use theTrainer()? The tutorial for continue training seems to be rare.I’m working onprajjwal1/bert-tiny(now) andbert-base-uncased(next step).
2022-05-25T11:32:54Z
yes, you load the model from .from_pretrained(), and use trainer to train. The link below has examples and video tutorialshuggingface.coFine-tune a pretrained modelWe’re on a journey to advance and democratize artificial intelligence through open source and open science.