Getting self-attention values of the GPT2LMHead model before softmax
Hi! I’m trying to get the alignment values used for the attention weights before the softmax in the GPTLMHead model for a research project. Does anyone know if this is possible/how to do it?I’m essentially trying to get the attention weights before this part of the model:transformers/src/transformers/models/gpt2/modeling_gpt2.py at 2a9b1f80c45cab19b542bc7cc004937d39d6f6fb · huggingface/transformers · GitHub.I would appreciate any help!