@inproceedings{rohanian-etal-2023-using,
    title = "Using Bottleneck Adapters to Identify Cancer in Clinical Notes under Low-Resource Constraints",
    author = "Rohanian, Omid  and
      Jauncey, Hannah  and
      Nouriborji, Mohammadmahdi  and
      Kumar, Vinod  and
      Gonalves, Bronner P.  and
      Kartsonaki, Christiana  and
      Clinical Characterisation Group, Isaric  and
      Merson, Laura  and
      Clifton, David",
    editor = "Demner-fushman, Dina  and
      Ananiadou, Sophia  and
      Cohen, Kevin",
    booktitle = "The 22nd Workshop on Biomedical Natural Language Processing and BioNLP Shared Tasks",
    month = jul,
    year = "2023",
    address = "Toronto, Canada",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2023.bionlp-1.5",
    doi = "10.18653/v1/2023.bionlp-1.5",
    pages = "62--78",
    abstract = "Processing information locked within clinical health records is a challenging task that remains an active area of research in biomedical NLP. In this work, we evaluate a broad set of machine learning techniques ranging from simple RNNs to specialised transformers such as BioBERT on a dataset containing clinical notes along with a set of annotations indicating whether a sample is cancer-related or not. Furthermore, we specifically employ efficient fine-tuning methods from NLP, namely, bottleneck adapters and prompt tuning, to adapt the models to our specialised task. Our evaluations suggest that fine-tuning a frozen BERT model pre-trained on natural language and with bottleneck adapters outperforms all other strategies, including full fine-tuning of the specialised BioBERT model. Based on our findings, we suggest that using bottleneck adapters in low-resource situations with limited access to labelled data or processing capacity could be a viable strategy in biomedical text mining.",
}

Processing information locked within clinical health records is a challenging task that remains an active area of research in biomedical NLP. In this work, we evaluate a broad set of machine learning techniques ranging from simple RNNs to specialised transformers such as BioBERT on a dataset containing clinical notes along with a set of annotations indicating whether a sample is cancer-related or not. Furthermore, we specifically employ efficient fine-tuning methods from NLP, namely, bottleneck adapters and prompt tuning, to adapt the models to our specialised task. Our evaluations suggest that fine-tuning a frozen BERT model pre-trained on natural language and with bottleneck adapters outperforms all other strategies, including full fine-tuning of the specialised BioBERT model. Based on our findings, we suggest that using bottleneck adapters in low-resource situations with limited access to labelled data or processing capacity could be a viable strategy in biomedical text mining.
Rohanian, Omid Jauncey, Hannah Nouriborji, Mohammadmahdi Kumar, Vinod Gonalves, Bronner P. Kartsonaki, Christiana Clinical Characterisation Group, Isaric Merson, Laura Clifton, David
Using Bottleneck Adapters to Identify Cancer in Clinical Notes under Low-Resource Constraints