Is the multiple-choice head for the pre-trained `LongformerForMultipleChoice` model pre-trained as well?
Hello,Is the multiple-choice head for the pre-trainedLongformerForMultipleChoicemodel pre-trained as well?I am asking because like for the pre-trainedGPT2DoubleHeadsModel, the main body of the model is trained but its multiple-choice head is not.Is the multiple-choice head for the pre-trainedLongformerForMultipleChoicemodel untrained, just like the case for theGPT2DoubleHeadsModel?Thank you,
2020-10-02T14:48:03Z
Hi@h56choYes , multiple-choice head is randomly initialized, you need to train these heads according to your  tasks.