Fine-tuning seq2seq transformer model for Arxiv research paper summarization in abstractive summarization manner
Hi Community,In my research area, I’m about tofine-tune the BART or T5 transformer modelfor the summarization of Arxiv research papers. For that purpose, I’m going to use acustom datasetwhich contains only Arxiv papers related to the Machine Learning domain. The custom dataset (includes abstract, article, section_names, sections columns) is a subset of the “Scientific Paper” dataset in Hugging Face (scientific_papers · Datasets at Hugging Face) which contains papers related to the Machine learning domain.My problem is I’m going to do the summarization in the following way as a novel approach,I’m about to give weightage to each section(such as importance, related work, methodology, etc) of the research paper based on their importance.In the paper summarization process sections with higher weightage will be more focused when extracting important data.I don’t know any specific way to calculate the above-mentioned weight as well as how to add that part when the transformer model fine-tuning.If you guys know any way to do that, I’d be really glad to hear. If you guys know how to do it, please provide the reference materials as well.