ByT5 for text classification
Hi!Facing a problem whilst trying to finetune the ByT5 model for the text classification task.I’ve tried to use this notebookexploring-T5/t5_fine_tuning.ipynb at master · patil-suraj/exploring-T5 · GitHubby@valhallafor ByT5 finetuning on the text classification task. When I start from ‘google/byt5-small’ I get really strange results. A model always generates negative sentiment label (‘n’ in my code)Screenshot 2021-07-30 at 13.04.301356×458 40.4 KBWhen I switch to t5-small pretrained checkpoint the results are reasonableScreenshot 2021-07-30 at 12.19.551356×458 41.4 KBWhat am I missing here? Any piece of advice would be much appreciated!
2022-06-20T05:37:38Z
Hello@janyfe. Could you find a solution for this? I am also facing same issue with multi-class text classification task. Among 6 classes, byt5 always predicts 1 class.
2022-06-20T11:56:32Z
Hello@Chandrai! No, unfortunately, I wasn’t able to find the solution and gave up using byt5 model.
2024-07-21T07:27:55Z
Hello@Chandrai. Did you find a solution?