Whitelist specific tokens in beam search
I’m usingmodel.generate()for text generation.I’m wondering is there a way to whitelist specific tokens so that they are returned during the beam search phase.For example - I want to the “force” the response to contain a question mark or a speccific phrase
2021-03-12T10:39:11Z
Every token is “whitelisted” in the sense that it is considered during beam search. However, you can modify/boost certain tokens you would like to see generated, using theLogitsProcessor. Just implement your own class that boosts your favourite tokens or sequence of tokens. If you are looking for a question mark - maybe you can “blacklist” other end of sentence punctuations in your LogitsProcessor, too?
2021-03-16T11:25:04Z
Thanks@100worte,I’m new to transformers and I’m still trying to understand how to use theLogitProcessor.Do I just manually augment the logitscoresreturned?How do pass my custom class intomodel.generate?
2021-03-16T19:13:18Z
You can pass aLogitsProcessorListtobeam_search. For some concrete examples ofLogitProcessors, refer togeneration_logits_process.py.
2021-11-11T16:54:36Z
Is it possible to pass customLogitsProcessortomodel.generate()?