text
📢 Have you ever been wondered how specifically Transformers were capable for handling long input contexts? 
📢 Have you ever been wondered how specifically Transformers were capable for handling long input contexts? 
new_line
text
I got a chance to tackle this through long document texts summarization problem, and delighted to share the related survey and diagram for a quick skimming below:
I got a chance to tackle this through long document texts summarization problem, and delighted to share the related survey and diagram for a quick skimming below:
new_line
new_line
text
Preprint 📝 
Preprint 📝 
link
https://nicolay-r.github.io/website/data/preprint-AINL_2023_longt5_summarization.pdf
https://nicolay-r.github.io/website/data/preprint-AINL_2023_longt5_summarization.pdf
new_line
text
Springer 📝 
Springer 📝 
link
https://link.springer.com/article/10.1007/s10958-024-07435-z
https://link.springer.com/article/10.1007/s10958-024-07435-z
new_line
new_line
text
🎯 The aim of the survey was the development of the long-document summarizer for mass-media news in Vietnamese language. 🇻🇳 
🎯 The aim of the survey was the development of the long-document summarizer for mass-media news in Vietnamese language. 🇻🇳 
new_line
new_line
text
Sharing for a quick skimming of the methods performance overview of various LM-based solution across several datasets, covering domain-oriented advances in Vietnamese language (see attached screenshots)
Sharing for a quick skimming of the methods performance overview of various LM-based solution across several datasets, covering domain-oriented advances in Vietnamese language (see attached screenshots)
new_line
new_line
text
As for solution we consider:
As for solution we consider:
new_line
text
☑️ 1. Adapt existed 
☑️ 1. Adapt existed 
resource
https://huggingface.co/google/pegasus-cnn_dailymail
model
google/pegasus-cnn_dailymail
https://huggingface.co/google/pegasus-cnn_dailymail
text
 for summarizing large dataset for arranging training
 for summarizing large dataset for arranging training
new_line
text
☑️ 2. Tuning 
☑️ 2. Tuning 
resource
https://huggingface.co/google/long-t5-tglobal-large
model
google/long-t5-tglobal-large
https://huggingface.co/google/long-t5-tglobal-large
text
  suitable for performing generative summarization.
  suitable for performing generative summarization.
new_line
new_line
text
Implementation details:
Implementation details:
new_line
text
🌟 
🌟 
link
https://github.com/nicolay-r/ViLongT5
https://github.com/nicolay-r/ViLongT5
new_line
text
(Simplier to go with huggingface rather flaxformer that so far become a legacy engine)
(Simplier to go with huggingface rather flaxformer that so far become a legacy engine)
new_line
📢 Have you ever been wondered how specifically Transformers were capable for handling long input contexts? 
I got a chance to tackle this through long document texts summarization problem, and delighted to share the related survey and diagram for a quick skimming below:

Preprint 📝 https://nicolay-r.github.io/website/data/preprint-AINL_2023_longt5_summarization.pdf
Springer 📝 https://link.springer.com/article/10.1007/s10958-024-07435-z

🎯 The aim of the survey was the development of the long-document summarizer for mass-media news in Vietnamese language. 🇻🇳 

Sharing for a quick skimming of the methods performance overview of various LM-based solution across several datasets, covering domain-oriented advances in Vietnamese language (see attached screenshots)

As for solution we consider:
☑️ 1. Adapt existed https://huggingface.co/google/pegasus-cnn_dailymail for summarizing large dataset for arranging training
☑️ 2. Tuning https://huggingface.co/google/long-t5-tglobal-large  suitable for performing generative summarization.

Implementation details:
🌟 https://github.com/nicolay-r/ViLongT5
(Simplier to go with huggingface rather flaxformer that so far become a legacy engine)

👀
John6666
👍
BuiDoan