How to get cross-attention values of T5?
Hello,I am trying to finetune the T5 model and need to get cross attention scores as well as self-attention scores. However when I set output_attentions=True, the model only returns self-attention values.Any idea how to get cross-attention values such as 6 elements with B,8,Tx,Ty ? (num_heads=8, num_layers=6)I am doing forward call on t5 modelt5 = T5ForConditionalGeneration.from_pretrained(“t5-small”)outputs = t5(input_ids=input_ids, labels=output_ids, use_cache=False, output_attentions=True, output_hidden_states=True)output returns 7 elementsoutputs[0]: lossoutputs[1]: input_embeddingsoutputs[2]: decoder_hiddens :  B,Ty,Houtputs[3]: decoder_self_attentions     # B,8,Ty,Tyoutputs[4]: encoder_last_hidden states ?outputs[5]: encoder_hiddens?outputs[6]: encoder_self_attentions # B,Tx, TxThanks a lot!
2020-10-09T03:34:45Z
Hello,I have been trying to get the cross-attention weights as well for the MarianMT model.I found out inside the source code that attention weights from the encoder-decoder layer wasn’t being included in the attentions being returned.I’m basing my assumptions from looking at the DecoderLayer classhereIf you do find a workaround , please share it on this thread !Thanks!
2020-10-09T17:56:14Z
Hello,In this pull request you can see the work-around for t5.https://github.com/huggingface/transformers/pull/7213#issuecomment-696340597So in your case, I guess if you make the following changes:https://github.com/huggingface/transformers/blob/master/src/transformers/modeling_bart.py#L460add the _ variable (which holds cross-attentions) to the returned attention value, you can get layer-wise cross attentions scores when output_attentions=True.https://github.com/huggingface/transformers/blob/master/src/transformers/modeling_bart.py#L484Hope it helps,Best,