Keep NSP head after BertForPretraining
I am pretraining with MLM + NSP (BertForPretraining), and finetuning with NSP (BertForNextSentencePrediction).Is there an elegant way which I can keep the NSP head from the pretrained model?Thanks!
2022-02-01T18:30:58Z
Validated thatusing BertForNextSentencePredictiondoes indeed keep the weights of NSP head ofBertForPreTraining