Which hidden states have the highest score in beam search?
Hello, I’m currently utilizing thetransformerslibrary for inference purposes. I have enabledoutput_hidden_statesby setting it toTrue, and I’ve specified anum_beamsvalue of4. Consequently, the shape ofhidden_statesis[n_beams, n_tokens]. However, I’m uncertain about identifying the specific beam fromn_beamsthat correlates with the highest-scoring inference result.Here is an example:hidden_states.shape == [4, 50]
output_tokens.shape == [50]
# hidden_states[0] <--> output_tokens ?
# hidden_states[1] <--> output_tokens ?
# hidden_states[2] <--> output_tokens ?
# hidden_states[3] <--> output_tokens ?