How do I finetune the backbone of bert-base-uncased?
Hey there, I am interested in finetuningbert-base-uncased, I’m talking about finetuning the backbone of the bert, not finetuning it for a specific task. i.e. I have a legal dataset and want to make a legal-bert-base-uncased.I found this tutorialFine-tune a pretrained model, but it focuses on finetuning a prediction head rather than the backbone weights.I would like tofinetune the backbone weights here, by dumping large corpus of texts from my domain,train a prediction head with a more limited dataset from my domainis that possible?