
# Biomedical Longformer (base)

This is a derivative model based on [microsoft/BiomedNLP-PubMedBERT-base-uncased-abstract](https://huggingface.co/microsoft/BiomedNLP-PubMedBERT-base-uncased-abstract) BERT model developed in the work "Fine-Tuning Large Neural Language Models for Biomedical Natural Language Processing" by [Tinn et al. (2021)](https://arxiv.org/abs/2112.07869). All model parameters where cloned from the original model, while the positional embeddings were extended by cloning the original embeddings multiple times following [Beltagy et al. (2020)](https://arxiv.org/abs/2004.05150) using a python script similar to this one (https://github.com/allenai/longformer/blob/master/scripts/convert_model_to_long.ipynb).
{"language": "en", "license": "cc-by-nc-sa-4.0", "tags": ["biomedical", "long-documents"], "pipeline_tag": "fill-mask"}
transformers pytorch longformer fill-mask biomedical long-documents en arxiv:2112.07869 arxiv:2004.05150 license:cc-by-nc-sa-4.0 autotrain_compatible endpoints_compatible region:us
TAGS
#transformers #pytorch #longformer #fill-mask #biomedical #long-documents #en #arxiv-2112.07869 #arxiv-2004.05150 #license-cc-by-nc-sa-4.0 #autotrain_compatible #endpoints_compatible #region-us 


# Biomedical Longformer (base)

This is a derivative model based on microsoft/BiomedNLP-PubMedBERT-base-uncased-abstract BERT model developed in the work "Fine-Tuning Large Neural Language Models for Biomedical Natural Language Processing" by Tinn et al. (2021). All model parameters where cloned from the original model, while the positional embeddings were extended by cloning the original embeddings multiple times following Beltagy et al. (2020) using a python script similar to this one (URL
# Biomedical Longformer (base)

This is a derivative model based on microsoft/BiomedNLP-PubMedBERT-base-uncased-abstract BERT model developed in the work "Fine-Tuning Large Neural Language Models for Biomedical Natural Language Processing" by Tinn et al. (2021). All model parameters where cloned from the original model, while the positional embeddings were extended by cloning the original embeddings multiple times following Beltagy et al. (2020) using a python script similar to this one (URL
TAGS
#transformers #pytorch #longformer #fill-mask #biomedical #long-documents #en #arxiv-2112.07869 #arxiv-2004.05150 #license-cc-by-nc-sa-4.0 #autotrain_compatible #endpoints_compatible #region-us 
 # Biomedical Longformer (base)

This is a derivative model based on microsoft/BiomedNLP-PubMedBERT-base-uncased-abstract BERT model developed in the work "Fine-Tuning Large Neural Language Models for Biomedical Natural Language Processing" by Tinn et al. (2021). All model parameters where cloned from the original model, while the positional embeddings were extended by cloning the original embeddings multiple times following Beltagy et al. (2020) using a python script similar to this one (URL
TAGS
#transformers #pytorch #longformer #fill-mask #biomedical #long-documents #en #arxiv-2112.07869 #arxiv-2004.05150 #license-cc-by-nc-sa-4.0 #autotrain_compatible #endpoints_compatible #region-us 
# Biomedical Longformer (base)

This is a derivative model based on microsoft/BiomedNLP-PubMedBERT-base-uncased-abstract BERT model developed in the work "Fine-Tuning Large Neural Language Models for Biomedical Natural Language Processing" by Tinn et al. (2021). All model parameters where cloned from the original model, while the positional embeddings were extended by cloning the original embeddings multiple times following Beltagy et al. (2020) using a python script similar to this one (URL