Use EncoderDecoder models for text summarization
Please read the topic categorydescriptionto understand what this is all aboutDescriptionMost of the availableTransformer modelsfor text summarization are only available for English documents. At the same time, there are now many pretrained BERT-like models in non-English languages. The goal of this project is to explore whether the[EncoderDecoderarchitecture](https://huggingface.co/transformers/model_doc/encoderdecoder.html?highlight=encoderdecoder#encoder-decoder-models) inTransformers can be used to create summarization models using just the pretrained weights of encoder-based models.Your task is to pick a pretrained encoder in a non-English language and train it to summarise texts in that language.Model(s)Seeherefor example models that people have fine-tuned using this architecture. You task is to create your very own model with this technique!DatasetsSearch forsummarizationdatasets on theHubto get an appropriate corpus for this taskChallengesText summarization is a tricky NLP task, so the performance obtained with these models may not match what is observed for their English couterparts (where much more data is available)Desired project outcomesCreate a Streamlit or Gradio app onSpacesthat can summarize a document in your chosen languageDon’t forget to push all your models and datasets to the Hub so others can build on them!Additional resourcesLeveraging Pre-trained Checkpoints for Sequence Generation Tasks[PAPER]Leveraging Pre-trained Language Model Checkpoints for Encoder-Decoder Models[BLOG POST]Examples of these models on the Hub by@mrm8488:https://twitter.com/mrm8488/status/1458475725565141001?s=20Discord channelTo chat and organise with other people interested in this project, head over to ourDiscordand:Follow the instructions on the#join-coursechannel. Then join one of the following channels:#encoder-decoder-eschannel (Spanish)Just make sure you comment here to indicate that you’ll be contributing to this project
2021-11-16T11:59:00Z
Interesting project, I am interested in a Spanish summarizer using Encoder-Decoder model. Anyone else interested in this approach?
2021-11-16T12:19:50Z
Hey@edumunozsala, cool to hear that you’re interesting in tackling this project! I’ve created a Discord channel (see topic description) in case you and others want to use it
2023-12-28T12:00:31Z
I am interested to train it for Persian (Farsi) language. how can I contribute?