@inproceedings{zhou-etal-2023-improving-transferability,
    title = "Improving the Transferability of Clinical Note Section Classification Models with {BERT} and Large Language Model Ensembles",
    author = "Zhou, Weipeng  and
      Afshar, Majid  and
      Dligach, Dmitriy  and
      Gao, Yanjun  and
      Miller, Timothy",
    editor = "Naumann, Tristan  and
      Ben Abacha, Asma  and
      Bethard, Steven  and
      Roberts, Kirk  and
      Rumshisky, Anna",
    booktitle = "Proceedings of the 5th Clinical Natural Language Processing Workshop",
    month = jul,
    year = "2023",
    address = "Toronto, Canada",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2023.clinicalnlp-1.16",
    doi = "10.18653/v1/2023.clinicalnlp-1.16",
    pages = "125--130",
    abstract = "Text in electronic health records is organized into sections, and classifying those sections into section categories is useful for downstream tasks. In this work, we attempt to improve the transferability of section classification models by combining the dataset-specific knowledge in supervised learning models with the world knowledge inside large language models (LLMs). Surprisingly, we find that zero-shot LLMs out-perform supervised BERT-based models applied to out-of-domain data. We also find that their strengths are synergistic, so that a simple ensemble technique leads to additional performance gains.",
}

Text in electronic health records is organized into sections, and classifying those sections into section categories is useful for downstream tasks. In this work, we attempt to improve the transferability of section classification models by combining the dataset-specific knowledge in supervised learning models with the world knowledge inside large language models (LLMs). Surprisingly, we find that zero-shot LLMs out-perform supervised BERT-based models applied to out-of-domain data. We also find that their strengths are synergistic, so that a simple ensemble technique leads to additional performance gains.
Zhou, Weipeng Afshar, Majid Dligach, Dmitriy Gao, Yanjun Miller, Timothy
Improving the Transferability of Clinical Note Section Classification Models with BERT and Large Language Model Ensembles