Fine-Tune a T5 for sentence fusion
Fine-Tuning a Seq2Seq model for sentence fusion in English.Sentence fusion is the task of joining several independent sentences into a single coherent text.E.g.: the sentences:Hillary goes to school. Hillary meets her friends at school.could be fused toHillary goes to school and meets her friends.Currently there is only one model on the hub for sentence fusion as can be seen on the dataset site:discofuse · Datasets at Hugging Face.The goal of this project is to have the best sentence fusion model for English on the hub.ModelOne use one or multiple of the pretrained T5 models:google/byt5-base · Hugging Facegoogle/t5-v1_1-base · Hugging Facet5-base · Hugging FaceDatasetsThe Discofuse dataset can be used:discofuse · Datasets at Hugging FaceAvailable training scriptsAs this will be a Seq2Seq model, therun_summarization_flax.pyscript can be used for training.(Optional) Desired project outcomeThe desired outcome is to have a sentence fusion model for the English language. This can be showcased directly on the hub or with a streamlit  or gradio app.(Optional) ChallengesBeating the existing model will be the most challenging part.(Optional) Links to read upon[1907.12461] Leveraging Pre-trained Checkpoints for Sequence Generation Tasksgoogle/roberta2roberta_L-24_discofuse · Hugging Facediscofuse · Datasets at Hugging Face
2021-06-25T16:08:40Z
Interested in this project.