Finetuning model with smaller sequence size and Dmodel
Hi all !I wanted to know if there is any way to finetune a model but with smaller parameters (for exemple : sequence length from 512 to 64 and dModel from 768 to 256).I could use the pytorch TransformerEncoder and train my own from scratch but I wanted to know if there were some fine-tunning possible (i.e not starting from scratch).Thanks in addvance for any help,Have a great day !