NLI 2-sentence classification with GPT2, XLNet, etc.?
I’m doing research on NLI with 2-sentence classification. I have already successfully used BERT and itsBertForSequenceClassificationclass to feed in two sentences in the form of an input string[CLS] sent1 [SEP] sent2 [SEP]and then perform classification.I’d like to do the same with the other models available, such as GPT2, XLNet, and RoBERTa. However, I can’t seem to find any example code that takes two sentences for those models. Do they use the same type of input string as BERT? Can someone point me to the relevant webpages for more information?Thank you for any help.
2020-09-08T18:04:05Z
Surely there must be someone who’s done NLI with one of these other non-BERT models?
2020-09-09T12:43:02Z
Hi@facehugger2020XLNetandRoBERTacan take sequence pairs just like BERT does. For roberta its’s encoded likes<s>sent1 </s></s> sent2</s>. If you pass the sentence pair to thetokenizerit’ll automatically encode it like that.You can use therun_glue.pyscripthereto fine-tuneXLNet,RoBERTaonNLIWIth GPT-2, you can useGPT2Modelclass (without LM head), and use the final embeddings returned by the model and pass them to the classification layer. Here’sone exampleI found.