T5ForConditionalGeneration, How to get prediction probabilities or logits at the inference time? (to calculate perplexity)
Hello. It is the situation exactly as described in the title.I want to get model’s prediction probabilities or logits to calculate perplexity.But as you know we can not get prediction probabilities if we usemodel.generate()method built in T5ForConditionalGeneration—we only can get prediction tokens, not probabilities.One possible method I think is to usemodel()method (forward). Because it returns logits and I can use those logits to calculate perplexity. But the additional unnecessary coding for decode logits should be followed. So I wanna know if there is any better solutions to calculate perplexity.So my question isHow to get prediction probabilities or logits at the inference time?orTeach me if there is other better way to calculate perplexity!Thanks a lot in advance