GPT-Neo text vs text_target for Seq2Seq Task
Hi there,I am finetuning a GPT-Neo model on a seq2seq task and am a bit confused about the details on how to fine-tune. Specifially, how to construct and tokenize:Do I pass input and target to the tokenizer together intokenizer(text=...)or should I usetextfor the input andtext_targetfor the target? In [1] it looks like you can just concatenate both and pass totext, but how can the model differentiate between input and target?Thanks[1]gpt-neo-fine-tuning-example/gpt_neo.py at main · dredwardhyde/gpt-neo-fine-tuning-example · GitHub