How to create a custom decoding strategy in the GenerationMixin class?
I am currently doing research on auto-regressive text generation, and I would like to add a custom decoding strategy in theGenerationMixinclass which can be called by:model.my_decoding_strategy(...)Any pointers (e.g., how to subclass GenerationMixin) would be much appreciated!@patrickvonplaten@sgugger
2023-02-15T10:47:30Z
Hi@wlchenFirst things first: you can implement a stand-alone decoding strategy and it will work (e.g. you can easily implement a stand-alonegreedy_search). Subclassing inGenerationMixinis not a hard requirement.The main advantage of subclassing would be the tight integration with.generate(), which does a lot of input preparation and checks under the hood. From.generate(), the individual decoding strategies are then called.My suggestion:Add a set of conditions to trigger your decoding strategy from.generate(), seeherefor examples.Copy-paste the existinggreedy_search(or whichever decoding strategy is the most similar with the one you’re trying to implement), and adapt from there
2023-02-16T15:41:33Z
Hi@joaoganteThanks for the clear and detailed explanation!I will try my implementation with your suggestion