When does the third conditional branch of the project function that generates hidden_states→query,value in T5 work?
It is unclear whenthe modeling_t5.py’s conditional branch on line 488is executed.Thekey_value_states, which is the first and second conditional branch, will contain the final output of the encoder at cross attention. Thus, we can see that it will be None otherwise, like the encoder’s forward.I understand thatthe last hidden_states = past_key_valueis a cache to omit the calculation during autoregressive.What, then, does theconditional branch 488andconcatinate on line 491do?