@inproceedings{singh-etal-2023-large,
    title = "Large Scale Sequence-to-Sequence Models for Clinical Note Generation from Patient-Doctor Conversations",
    author = "Singh, Gagandeep  and
      Pan, Yue  and
      Andres-Ferrer, Jesus  and
      Del-Agua, Miguel  and
      Diehl, Frank  and
      Pinto, Joel  and
      Vozila, Paul",
    editor = "Naumann, Tristan  and
      Ben Abacha, Asma  and
      Bethard, Steven  and
      Roberts, Kirk  and
      Rumshisky, Anna",
    booktitle = "Proceedings of the 5th Clinical Natural Language Processing Workshop",
    month = jul,
    year = "2023",
    address = "Toronto, Canada",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2023.clinicalnlp-1.18",
    doi = "10.18653/v1/2023.clinicalnlp-1.18",
    pages = "138--143",
    abstract = "We present our work on building large scale sequence-to-sequence models for generating clinical note from patient-doctor conversation. This is formulated as an abstractive summarization task for which we use encoder-decoder transformer model with pointer-generator. We discuss various modeling enhancements to this baseline model which include using subword and multiword tokenization scheme, prefixing the targets with a chain-of-clinical-facts, and training with contrastive loss that is defined over various candidate summaries. We also use flash attention during training and query chunked attention during inference to be able to process long input and output sequences and to improve computational efficiency. Experiments are conducted on a dataset containing about 900K encounters from around 1800 healthcare providers covering 27 specialties. The results are broken down into primary care and non-primary care specialties. Consistent accuracy improvements are observed across both of these categories.",
}

We present our work on building large scale sequence-to-sequence models for generating clinical note from patient-doctor conversation. This is formulated as an abstractive summarization task for which we use encoder-decoder transformer model with pointer-generator. We discuss various modeling enhancements to this baseline model which include using subword and multiword tokenization scheme, prefixing the targets with a chain-of-clinical-facts, and training with contrastive loss that is defined over various candidate summaries. We also use flash attention during training and query chunked attention during inference to be able to process long input and output sequences and to improve computational efficiency. Experiments are conducted on a dataset containing about 900K encounters from around 1800 healthcare providers covering 27 specialties. The results are broken down into primary care and non-primary care specialties. Consistent accuracy improvements are observed across both of these categories.
Singh, Gag eep Pan, Yue Andres-Ferrer, Jesus Del-Agua, Miguel Diehl, Frank Pinto, Joel Vozila, Paul
Large Scale Sequence-to-Sequence Models for Clinical Note Generation from Patient-Doctor Conversations