PreTrain GPT2 from scratch in Punjabi
GPT2 for PunjabiPretrain GPT2 on Punjabi language to create a strong language generation model for PunjbaiModelA randomly initialized GPT2 modelDatasetsOne can make use of Kaggle Wikipedia Punjabi Articles dataset -Punjabi Wikipedia Articles | KaggleAvailable training scriptsA causal language modeling script for Flax is availablehere.(Optional) Desired project outcomeThe desired project output is a GPT2 model that is able to generate Punjabi language.
2021-06-25T18:02:59Z
Hey@hgarg!Great to see someone working on Punjabi. There is definitly a lack of models for Punjabi in the HuggingFace Hub, so this can definitly supplement that.Great idea and all the best!
2021-06-29T14:39:29Z
Cool idea@bhavnicksm- would you like to join?