Obtain step by step outputs of model.generate
Hi,I am usingLEDForConditionalGenerationfor fine-tuning a summarization model. I am able to finetune the network and obtain good results for my purpose. But now, I want to modify the encoder to use some different architecture. How can I break themodel.generate()into individual components of the transformer?For example, I want something like this:outputs = model(input_ids)
encoder_outputs = outputs["encoder_last_hidden_state"]  #For encoder outputs
# Similar logic for decoder outputs and then beam search to get text sentencesHow can I obtain such encoder decoder outputs and obtain exactly the same result that I get withmodel.generate? A sample code formodel.generateis as followsARTICLE_TO_SUMMARIZE = "Some long document that I want to summarize.
inputs = tokenizer.encode(ARTICLE_TO_SUMMARIZE, return_tensors="pt")
global_attention_mask = torch.zeros_like(inputs)
global_attention_mask[:, 0] = 1
# Generate Summary
summary_ids = model.generate(inputs, global_attention_mask=global_attention_mask, num_beams=3, max_length=32)
print(tokenizer.decode(summary_ids[0], skip_special_tokens=True, clean_up_tokenization_spaces=False))Any help in this direction will be appreciated. Thanks!