Fine Tuning GPT2 for machine translation
good evening everyone,is it possible to fine-tune gpt2 for text translation?if it is possible, how can I do it using my own data?I want to translate from ASL to English, and the idea that came to me was to use gpt2 as the decoder (since it is trained in English) and use a BERT as an encoder (I would fine tune it and retrain with the ASL base)Does anyone have a tutorial on how to do something like this?
2021-05-02T10:22:49Z
hey@yansoares, you could try using theEncoderDecoderModel(docs) to create a BERT2GPT model. I’m not aware of any tutorials using this class, but the original paper (link) is quite well written and should give you a good idea on what needs to be done.