Question about greedy_search
Hello. I am trying togreedy_searchwithT5ModelForConditionalGeneration.However, it seems I have to provideinput_idsas both theinput_idsparameter and as part ofmodel_kwargsparameter of thegreedy_search.Theinput_idsargument ofgreedy_searchacts as the initial decoded state, whileinput_idsthat is supposed to appear inmodel_kwargsis passed toself(T5) for inference.I do not see a way to pass twoinput_idsargument to thegreedy_searchfunction. Right now the function complains that I don’t passinput_idsto its internal T5.How would I go about using thegreedy_searchfunction fotT5?
2021-04-26T01:17:04Z
Solved by passingencoder_outputstogreedy_seach.
2021-06-18T04:11:47Z
Hi seongmin, I am facing the same issue currently but I am not sure what to do. Would you have a code sample to share on how you managed to pass theencoder_outputstogreedy_search? Thank you.
2021-06-18T04:31:41Z
Hi Reuben.While I can’t share the exact code because I don’t own it, here’s the gist of what I did:model = T5ForConditionalGeneration(...)
encoder_outputs = model.encoder(encoder_input_ids, return_dict=True, output_hidden_states=True)
decoder_input_ids = torch.tensor([t5config.starter_token_id])
generated = model.greedy_search(decoder_input_ids, encoder_outputs=encoder_outputs)
2021-06-18T05:14:03Z
Hi@seongmin,That was immensely helpful thank you so much! Have a great day