Seeking clarification on MPNet fine-tuning objectives
Hi there,I am interested in fine-tuning the MPNet pre-trained model on a domain-specific corpus. I understand that the MPNet model combines both autoregressive and autoencoding language modeling objectives. Does therun_mlm.pyorrun_mlm_wwm.pyscript with the MPNet model as argument trains with both objectives or the masked LM objective as the name of the script implies?