Ensure the sentence is complete during generation
During generation, I’m using the constraint ofmax_lengthto stop if longer sequences are not required. However, I do not want the generation to stop if the sentence is not complete. Is there any reliable way to stop after one sentence has been generated ?
2021-03-17T17:38:49Z
AFAIK, the generation should stop once it generates an end-of-sentence token, if youdon’tspecifymax_length. You can useStoppingCriteria(which you implicitly do by settingmax_length) to construct arbitrary constraints on when to stop your generation.
2021-03-18T02:52:28Z
You’re right about EOS token. If I don’t specifymax_lengthparameter, then the model can generate a long text which may stop making sense halfway through or deviates from the context provided. I want the generation to be a bit more natural. Can you please share an example of howStoppingCriteriawould work ? Didn’t find the usage example in docs.
2021-03-19T07:41:34Z
You can find some implementationshere. And you can search for “stopping_criteria” ingeneration_utils.pyto understand the usage.
2023-04-18T19:20:12Z
I was struggling with this issue today. The easiest way (instead of providing a custom stopping criteria) is to set bothmin_length and max_lengthparameters to the same value. This ensures generations of exactly a given length, no shorter or longer.