Create a SentenceTransformer in Dhivehi using ELECTRA
DescriptionDhivehi is a low resource language. Since the available data is less, ELECTRA seems to be a good option as it requires less computing power and training data as compared to others.Modelelectra-small pretrained in dhivehi availablehereDiscord channelTo chat and organise with other people interested in this project, head over to ourDiscordand:Follow the instructions on the#join-coursechannelJoin the#sentence-transformers-dhivehichannelJust make sure you comment here to indicate that you’ll be contributing to this project
2021-11-17T16:21:17Z
He@ashraqthanks for proposing this interesting project! One question: what do you mean by create a “sentence transformer”? Are you talking about adding a pooling layer to theelectra-smallmodel and then training that on a Dhivevi corpus?Do you also happen to have access / know of a Dhivevi corpus to train on?
2021-11-17T17:26:36Z
Yes. Adding a pooling layer and fine tuning sentence transformer for dhivehi.Yes there is a corpus available. In fact i have been preparing data for this task since last week before I came to know about the event. So i think this will be a wonderful opportunity.
2021-11-17T21:09:38Z
This sounds like a great project indeed! I’ve created a Discord channel (see topic description) in case you and others want to use it
2021-11-17T21:43:28Z
btw are there any limitations on the instance we can choose on aws sage maker during this event?
2021-11-17T22:01:06Z
As far as I know you can choose a p3 instance if it’s available or a T4 if not