Generation but constraining first few tokens
I’m trying to use a pre-trained T5ForConditionalGeneration model to do generation from a prefix of a sentence. For example, I might feed it “   2 + 2 =”, and I’d like it to generate “4”.I notice that the generate() function for this model always produces a generation starting with the two tokens [pad_token, bos_token]. Therefore, if I just pass the input through model.generate, I get a nonsensical output. I see two potential solutions:Feed it in “”, and fix the first few tokens of the generation to be “2 + 2 =”Feed it in “   2 + 2 =”, and find a way to remove the pad and bos tokens from the generation.I don’t know how to do either, and help would be appreciated! Thanks!