Image captioning for Indonesia with pre-trained vision
Image captioning for Indonesia with pre-trained vision and text modelFor this project, a pre-trained image model likeViTcan be used as an encoder, and a pre-trained text model like BERT and/or GPT2 can be used as a decoder.ModelPre-trainedViT,BERTmodels can be found on the model hub. We could use multi-lingual BERT/ROERTa models for the Indonesian language.DatasetsTheWITdataset can be used for this task. It has almost over 200K image-text pairs for Indonesia.TheGEMdataset can also be used for the taskAvailable training scriptsAs this will be a Seq2Seq model, therun_summarization_flax.pyscript can be used for training this model with some modifications.(Optional) Desired project outcomeThe desired outcome is to see if pre-trained vision and text models can be leveraged for image captioning and also train captioning models in for the Indonesia language. This can be showcased with a streamlit or gradio app.(Optional) Challenges-This model will require some modifications to the existing text models.-Data Processing
2021-06-24T23:16:53Z
Count me in!
2021-06-25T19:05:56Z
I am also in
2021-06-29T09:28:41Z
I am also in
2021-06-29T15:16:39Z
Great! Let’s define the projectcc@valhalla