Finetuning BART on a multi-input sequence to sequence task
I finetuned bart-base on a sequence to sequence task and I have the following questions:a) Currently I structured the input and output for the bart model in “t5-style” by adding prefixes in front of each piece of input. For bart how should I give multiple inputs (or train it to return multiple outputs) to the model (is there special token to separate inputs, should I continue the t5-style prefixes, etc.)? Also, how would I do this for gpt-2/gpt-neo?b) When finetuned with prefixes, the target data is formatted with “output: …”, however, the finetuned-bart returns “outputoutput: …”. Why is this repetiton occurring? Also, does the Bart tokenizer automatically add the eos token?c) Also does the trainer api automaticallly handleadjust_logits_during_generationanddecoder_start_token_idas discussed in thispost?@valhallaCould you please help with this? This is my first project training an nlp model, and I would really appreciate any information you can offer regarding my questions.