TFT5ForConditionalGeneration generate returns empty output_scores
Hi everyone,I’m fine-tuning the TFT5ForConditionalGeneration model (“t5-small”). Before doing model.fit() and save the model I set the output_score=True as well as the relevant parameters in the generation config. After fine-tuning the model I load it and I can see that in the loaded model theconfig.output_scores = Trueas well as thegeneration_config.output_scores=True.However, when I generate the text, thescoresreturns empty.How can I fix this please? I read documentationhttps://huggingface.co/docs/transformers/internal/generation_utils#transformers.generation.TFSampleEncoderDecoderOutput,https://huggingface.co/docs/transformers/v4.36.1/en/main_classes/text_generation#transformers.TFGenerationMixinbut wasn’t able to figure out how to get the scores in the generated text.I’m using  transformers version 4.36.0from transformers import TFAutoModelForSeq2SeqLM, GenerationConfig, AutoConfig

config = AutoConfig.from_pretrained("t5-small")
config.output_scores = True
transformer_model = TFAutoModelForSeq2SeqLM.from_pretrained("t5-small", config=config)

generation_config = GenerationConfig(**transformer_model.generation_config.__dict__)
generation_config.max_new_tokens = 10
generation_config.return_dict_in_generate = True
generation_config.output_scores = True

transformer_model.save_pretrained(transformer_model_path)
generation_config.save_pretrained(transformer_model_path)

# text generation
input_ids = [[283, 1890, 6, 2733, 18, 715, 621, 29, 32, 106, 4804, 89, 17, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]]
generated_text = transformer_model.generate(input_ids, generation_config=generation_config, return_dict_in_generate=True, output_scores=True)

print(generated_text.scores)  # returns empty tuple