@inproceedings{chowdhury-etal-2023-large,
    title = "Can Large Language Models Safely Address Patient Questions Following Cataract Surgery?",
    author = "Chowdhury, Mohita  and
      Lim, Ernest  and
      Higham, Aisling  and
      McKinnon, Rory  and
      Ventoura, Nikoletta  and
      He, Yajie  and
      De Pennington, Nick",
    editor = "Naumann, Tristan  and
      Ben Abacha, Asma  and
      Bethard, Steven  and
      Roberts, Kirk  and
      Rumshisky, Anna",
    booktitle = "Proceedings of the 5th Clinical Natural Language Processing Workshop",
    month = jul,
    year = "2023",
    address = "Toronto, Canada",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2023.clinicalnlp-1.17",
    doi = "10.18653/v1/2023.clinicalnlp-1.17",
    pages = "131--137",
    abstract = "Recent advances in large language models (LLMs) have generated significant interest in their application across various domains including healthcare. However, there is limited data on their safety and performance in real-world scenarios. This study uses data collected using an autonomous telemedicine clinical assistant. The assistant asks symptom-based questions to elicit patient concerns and allows patients to ask questions about their post-operative recovery. We utilise real-world postoperative questions posed to the assistant by a cohort of 120 patients to examine the safety and appropriateness of responses generated by a recent popular LLM by OpenAI, ChatGPT. We demonstrate that LLMs have the potential to helpfully address routine patient queries following routine surgery. However, important limitations around the safety of today{'}s models exist which must be considered.",
}

Recent advances in large language models (LLMs) have generated significant interest in their application across various domains including healthcare. However, there is limited data on their safety and performance in real-world scenarios. This study uses data collected using an autonomous telemedicine clinical assistant. The assistant asks symptom-based questions to elicit patient concerns and allows patients to ask questions about their post-operative recovery. We utilise real-world postoperative questions posed to the assistant by a cohort of 120 patients to examine the safety and appropriateness of responses generated by a recent popular LLM by OpenAI, ChatGPT. We demonstrate that LLMs have the potential to helpfully address routine patient queries following routine surgery. However, important limitations around the safety of today{'}s models exist which must be considered.
Chowdhury, Mohita Lim, Ernest Higham, Aisling McKinnon, Rory Ventoura, Nikoletta He, Yajie De Pennington, Nick
Can Large Language Models Safely Address Patient Questions Following Cataract Surgery?