Prohibit GPT-2 from generating some words on a condition
Hello,I have added to the GPT-2 vocabulary two special tokens ([ss],[se]) and use it to generate sequences.I want to prohibit GPT-2 from generating some words after having generated[ss]until it generates[se]. For example in the following sequencesome words of the sequence [ss] here I want some words not to be generated [se] other words of the sequence.in other words, I want between[ss]…[se]to exclude some words from the generation. As@deathcrushansweredhereit is possible to prohibit GPT-2 from generating some vocabulary ids in general, but is it possible to do it on condition? (only between those tokens[ss]and[se])
2021-03-21T14:04:05Z
Could you check out this function for contained prefix generation:Models — transformers 4.4.2 documentation
2021-03-21T16:03:21Z
Thank you@patrickvonplatenfor your fast reply.The link you provided is for the models page. Could you point me to the function you are talking about?
2021-03-22T19:38:29Z
@patrickvonplatenI saw thatgeneratefunction has aprefix_allowed_tokens_fn. Were you referring to this?
2021-03-23T02:44:56Z
Yes, that is what he meant.
2021-03-23T21:06:01Z
This function is used to constrainbeam search. I want to generate text withsamplingand specifically withtop_p. Is there a way to utilise this function?
2021-03-29T17:47:06Z
@patrickvonplatenin the documentation ofgeneratefunction the description ofprefix_allowed_tokens_fnargument saysIf provided, this function constraints the beam search to allowed tokens only at each step.Can  this function be applied with sampling as well?
2021-04-25T18:03:02Z
hfnlpmb:tgeneratefunction has aprefix_allowed_tokens_fn. Were you referring to this?Hey@hfnlpmb,yes the function can be used in combination withdo_sample=True