Pretrained GPT2 for Tamil
GPT2 for TamilWe aim to create a GPT2 model for tamil language.ModelA randomly initialized GPT2 modelDatasetsOSCAR dataset (https://oscar-corpus.com/) has about 9 GB of training data and Indic-NLP (IndicCorp | AI4Bharat IndicNLP) has 500 million token dataset.Available training scriptsA causal language modeling script for Flax is available [here ]It can be used pretty much without any required code changes.(Optional) Desired project outcomeThe desired project output is a GPT2 model that is able to generate Tamil language.
2021-07-01T03:17:51Z
@rays2pix: Hi Deepak. I was hoping that someone would propose a project for Tamil. I would like to collaborate on this!
2021-07-01T03:47:59Z
That’s nice@abinayam. We can do this if this gets picked . What do you think@patrickvonplaten
2021-07-01T07:41:53Z
I cannot help much, but I would be excited to see this happen!
2021-07-01T09:44:40Z
Awesome! Let’s officially define it
2021-07-01T12:47:15Z
Great.@abinayamlet’s discuss on discord#gpt-tamil@100worte.
2021-07-01T12:56:24Z
Link for discordFlax-HuggingFace-Community-Week
2021-07-01T14:18:04Z
I cannot contribute to it. But wish you all the best!
2021-07-01T16:39:54Z
Thanks@rays2pixfor introducing me to this post . I would like to collaborate if there is room
2021-07-01T16:41:38Z
@adithya1111: You’re welcome to join the group. Please join the discord channel.
2021-07-11T19:02:38Z
Hi@abinayamthough I may not be able to contribute, I  would like to join the discussion in the discord channel, could you please share the link?
2021-07-12T04:52:46Z
Hi@cishwarya: Here is the channel name in discord#gpt-tamil.
2021-07-12T06:19:45Z
Hi@abinayamthanks for your reply. I do not have the invite for the discord group. Could you please send me an invite…thank you so much…
2021-07-12T07:31:43Z
Here is the invite for the discord group:Flax-HuggingFace-Community-Week