Greedy decoding produces empty output
I’ve finetunedctrl, and it seems that greedy decoding withrepetition_penalty=1.2andtemperature=0and top_k sampling('max_length': 64, 'do_sample':True, 'max_length':64, 'top_k':50, 'top_p':0.95)produces empty sequences often when varied examples are provided as context. I wanted to know what is the reason for this usually ? I tried playing with parameters inmodel.generate, but they didn’t seem to help much directly. Also, some checkpoints in between are able to generate text for that sample. What is the reason usually ?
2022-06-29T02:22:33Z
Hi I have same questionDid you figure out why this problem occurs !?
2023-11-22T04:09:06Z
Maybe need some format for prompt, such as decorate the context with question-answer format of f’Q:{context}\nA:’ to let finetuned model to generate the text you want.