@inproceedings{wang-etal-2023-check-covid,
    title = "Check-{COVID}: Fact-Checking {COVID}-19 News Claims with Scientific Evidence",
    author = "Wang, Gengyu  and
      Harwood, Kate  and
      Chillrud, Lawrence  and
      Ananthram, Amith  and
      Subbiah, Melanie  and
      McKeown, Kathleen",
    editor = "Rogers, Anna  and
      Boyd-Graber, Jordan  and
      Okazaki, Naoaki",
    booktitle = "Findings of the Association for Computational Linguistics: ACL 2023",
    month = jul,
    year = "2023",
    address = "Toronto, Canada",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2023.findings-acl.888",
    doi = "10.18653/v1/2023.findings-acl.888",
    pages = "14114--14127",
    abstract = "We present a new fact-checking benchmark, Check-COVID, that requires systems to verify claims about COVID-19 from news using evidence from scientific articles. This approach to fact-checking is particularly challenging as it requires checking internet text written in everyday language against evidence from journal articles written in formal academic language. Check-COVID contains 1, 504 expert-annotated news claims about the coronavirus paired with sentence-level evidence from scientific journal articles and veracity labels. It includes both extracted (journalist-written) and composed (annotator-written) claims. Experiments using both a fact-checking specific system and GPT-3.5, which respectively achieve F1 scores of 76.99 and 69.90 on this task, reveal the difficulty of automatically fact-checking both claim types and the importance of in-domain data for good performance. Our data and models are released publicly at \url{https://github.com/posuer/Check-COVID}.",
}

We present a new fact-checking benchmark, Check-COVID, that requires systems to verify claims about COVID-19 from news using evidence from scientific articles. This approach to fact-checking is particularly challenging as it requires checking internet text written in everyday language against evidence from journal articles written in formal academic language. Check-COVID contains 1, 504 expert-annotated news claims about the coronavirus paired with sentence-level evidence from scientific journal articles and veracity labels. It includes both extracted (journalist-written) and composed (annotator-written) claims. Experiments using both a fact-checking specific system and GPT-3.5, which respectively achieve F1 scores of 76.99 and 69.90 on this task, reveal the difficulty of automatically fact-checking both claim types and the importance of in-domain data for good performance. Our data and models are released publicly at \url{https://github.com/posuer/Check-COVID}.
Wang, Gengyu Harwood, Kate Chillrud, Lawrence Ananthram, Amith Subbiah, Melanie McKeown, Kathleen
Check-COVID: Fact-Checking COVID-19 News Claims with Scientific Evidence