Text generation pipeline - output_scores parameter
Intext-generationpipeline, I am looking for a parameter which calculates the confidence score of the generated text. Source:hereI am assuming that,output_scores(fromhere) parameter is not returned while prediction,Code:predictedText = pipeline('text-generation',model=checkpoint_path, tokenizer=gpt2_tokenizer, config={'max_length':20, 'output_scores':True})predictedText('This is a ')Output:Settingpad_token_idtoeos_token_id:50256 for open-end generation. [{'generated_text': 'This is a Generated Text'}]In the output, I am looking for a confidence score of the predicted text to be displayed
2021-01-20T07:24:55Z
thetext-generationpipeline doesn’t return scores, however you could thegeneratemethod directly, to get the scores, this should helpGeneration Probabilities: How to compute probabilities of output scores for GPT2🤗TransformersNow that it is possible to return the logits generated at each step, one might wonder how to compute the probabilities for each generated sequence accordingly. 
The following code snippet showcases how to do so for generation with do_sample=True for GPT2: 
import torch
from transformers import AutoModelForCausalLM
from transformers import AutoTokenizer


gpt2 = AutoModelForCausalLM.from_pretrained("gpt2", return_dict_in_generate=True)
tokenizer = AutoTokenizer.from_pretrained("gpt2")

input_ids …