T5/mT5 model distillation
Hi Everyone,I am trying to distill my T5 model. I am planning to use this script:transformers/distillation.py at main · huggingface/transformers · GitHubCan anyone please guide me how to setup my codebase for this process? If anyone have any other better solution please guide me.
2023-12-25T06:19:43Z
I am also trying to distill from T5-xxl. What teacher model did you choose to use?