Hugging Face's logo

rttl-ai
/
BIOptimus

Fill-Mask
Transformers
PyTorch
English
bert
biology
medical
Inference Endpoints
Model card Files Community
3
BIOptimus / README.md
rttl's picture
rttl
Update README.md
1efde63
preview code
|
raw
history blame
1.12 kB
metadata
license: apache-2.0
datasets:
  - pubmed
language:
  - en
tags:
  - biology
  - medical

rttl-ai/BIOptimus v.0.4

Model Details

Model Description: BIOptimus v.0.4 model pretrained on PubMed abstracts. It is introduced in the paper BIOptimus: Pre-training an Optimal Biomedical Language Model with Curriculum Learning for Named Entity Recognition (BioNLP workshop @ ACL 2023). More information is available in this repository.

This model achieves state-of-the-art performance on several biomedical NER datasets from BLURB.

  • Developed by: rttl-ai
  • Model Type: Text Classification
  • Language(s): English
  • License: Apache-2.0
  • Resources for more information:
  • The model was pre-trained with task-adaptive pre-training TAPT with an increased masking rate, no corruption strategy, and using WWM, following this paper
    • fine-tuned on sst with subtrees
    • fine-tuned on sst2