Support for LLaMA in EncoderDecoder framework
I’m trying to use LLaMA as a drop-in replacement for GPT2 in my ViT-GPT2 model.After seeing issueUsing FNet model in Encoder Decoder Models · Issue #22308 · huggingface/transformers · GitHub, it seems like HuggingFace doesn’t plan to support future models in the EncoderDecoder framework and I should adapt the model to suit my own needs.I’m planning to follow the steps described inTrying to add support for GPT2 as decoder in EncoderDecoder model · Issue #4483 · huggingface/transformers · GitHubAre there any gotchas I should know about?