Fine Tuning Git Model for Malayalam Image Captioning
Hello everyone,I am currently working on a project to fine-tune the GIT model for image captioning in the Malayalam language. I am following this tutorialTransformers-Tutorials/Fine_tune_GIT_on_an_image_captioning_dataset.ipynb at master · NielsRogge/Transformers-Tutorials · GitHubwhich fine-tunes the model on English captions. However, I am unsure about the changes that I need to make for Malayalam.Here are some of my questions:The tutorial uses an English tokenizer. What should I use for Malayalam text? Is there a pre-trained tokenizer available that I can use, or do I need to train my own?During model training, how can I ensure that the model’s output vocabulary matches the vocabulary of the Malayalam tokenizer?Are there any Malayalam-specific considerations I should take into account when evaluating the model’s performance?I would really appreciate any guidance or resources that can help me with this task. Thank you in advance for your time and help!