GPT2 Conditional Text Generation
Hello,I would like to fine-tune the GPT2 model on EmpatheticDialogues doing kind of conditional generation as like in this paper:https://arxiv.org/pdf/1911.11161.pdfWhat concerns me is the format of the input_ids and labels in the forward function.I think that concatenating the input with the target is a good solution separating them with a special token(e.g. "hi! how are you? I am fine!)However I am not sure what to do with the labels. Shall I mask all the input part and the padded tokens with -100 index and leave only the target part as is? or shall I mask with -100 only the padded tokens?Thank you in advance