Is this the right way prompt summarization with BART?
Hello!I recently figured out a way to prompt the summarization of Encoder/Decoder models like BART using thegenerate()function. Normally the only prompting we get with this function appears to be the starting token for the generation (decoder_start_token_id).However, we can see that theGenerationMixindoes accept a**kwargswhich gets forwarded to the underlying model!In BART’s case, we have another parameter to its forward,decoder_input_ids, which looks like it allows you to set a custom Q vector for the decoder, for that function call.Using these together, we can appear to makegenerate()start the beam search given the encoded input,and the prompt to the decoder:Screen Shot 2022-05-25 at 09.56.482394×1382 378 KBMy question is: Is this process doing what I think it is? The results seem to make sense, and I can loosely verify this method by prompting with only"<s>", which gives identical output to the defaults forgenerate(). Is this a good way to prompt summarization for the BART model?Thanks in advance!
2023-03-18T21:05:04Z
It seems you did have an excellent way of using prompts, according to your example. I am new to this and like to hear do you have any new progress on this?