Pre-training/fine-tuning Seq2Seq model for spelling and/or grammar correction in English
Pre-training/fine-tuning Seq2Seq model for spelling and/or grammar correctionFor this project, one can use a randomly initialized or a pre-trained BART/T5 model.ModelPre-trainedBART,T5models can be found on the model hub.DatasetsThe dataset for this model can be prepared as described in thisblog post.One can make useOSCAR. The dataset is also available through thedatasetslibrary here:oscar · Datasets at Hugging Face.Available training scriptsAs this will be a Seq2Seq model, therun_summarization_flax.pyscript can be used for training.(Optional) Desired project outcomeThe desired outcome is to train a spelling correction model for the English language. This can be showcased directly on the hub or with a streamlit  or gradio app.(Optional) ChallengesImplementing the dataset noising function would be the challenging part of the project.(Optional) Links to read uponMicrosoft Research – 8 Feb 21Speller100 expands spelling correction technology to 100+ languagesResearchers in Microsoft Search and AI created Speller100 to expand spelling correction technology to 100-plus languages. Learn how it uses the concept of language families and zero-shot learning to make spelling correction more inclusive of...
2021-06-23T18:04:02Z
I have been working on automatic punctuation prediction so I am keen to explore other aspects of language correction as well. This project would be great for that. Count me in!
2021-06-24T20:38:41Z
This sounds very interesting. Count me in!
2021-06-29T14:46:27Z
great finalizing the project
2021-07-04T11:29:04Z
Did you guys open a discord channel? Or do you have other means of communication?@naruto7- feel free to go ahead and open a discord channel if no one has done it yet
2021-09-13T19:46:03Z
Hello@valhalla.Did you publish a model and/or a notebook/script for this spelling correction model? Thanks.
2021-09-14T06:31:16Z
we did not get team to do this project, so sadly, no.
2021-10-11T17:16:30Z
Hi@valhalla.Just to say that I have continued to search this topic for spelling corrections (in particular, about ASR errors correction) and just posted informations that may be of interest to people of similar interest.ASR spell correctionResearchHi. 
When searching for solutions about ASR errors corrections, I found this topic in the HF forum. 
I would like to discuss with you about 2 models.FastCorrect
Recently, Microsoft Asia published FastCorrect paper (and more recently, FastCorrect 2). I like the 2 main ideas on which is based this model: 

Training of an edit distance model in order to adapt the tokens number of the source (sentence with errors from the ASR output) to the one of the target (sentence without errors): thus, the d…