Decode whisper logits to transcript using forward instead of generate method
I’m trying to use whisper to generate transcription. I can get results when usingmodel.generate(). But when I try to useout = model(inputs, decoder_input_ids = torch.tensor([[50258]]).to('cuda'))
    predicted_ids = torch.argmax(out.logits, dim=-1)
    transcription = processor.batch_decode(predicted_ids)
    transcription
    ['<|startoftranscript|>']I’m only getting one token output. Is the issue withdecoder_input_ids? I want to use foward because I want useencoderanddecoderembeddings for other tasks. Is there any workaround here?
2022-12-09T06:55:47Z
We can gethidden_states_embeddingsfrommodel.generateby enabling the two flags, output_hidden_states=True, return_dict_in_generate=True.
2022-12-16T09:12:58Z
Is it possible to access logits by using model.generate?However it is possible to get logits from model but as you mentioned, I couldn’t get more than one token.
2022-12-20T09:30:38Z
You can get scores in themodel.generatefunction usingoutput_scores=Trueand then applysoftmaxto get individual tokens logits.