@inproceedings{lee-etal-2023-square,
    title = "{SQ}u{AR}e: A Large-Scale Dataset of Sensitive Questions and Acceptable Responses Created through Human-Machine Collaboration",
    author = "Lee, Hwaran  and
      Hong, Seokhee  and
      Park, Joonsuk  and
      Kim, Takyoung  and
      Cha, Meeyoung  and
      Choi, Yejin  and
      Kim, Byoungpil  and
      Kim, Gunhee  and
      Lee, Eun-Ju  and
      Lim, Yong  and
      Oh, Alice  and
      Park, Sangchul  and
      Ha, Jung-Woo",
    editor = "Rogers, Anna  and
      Boyd-Graber, Jordan  and
      Okazaki, Naoaki",
    booktitle = "Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)",
    month = jul,
    year = "2023",
    address = "Toronto, Canada",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2023.acl-long.370",
    doi = "10.18653/v1/2023.acl-long.370",
    pages = "6692--6712",
    abstract = "The potential social harms that large language models pose, such as generating offensive content and reinforcing biases, are steeply rising. Existing works focus on coping with this concern while interacting with ill-intentioned users, such as those who explicitly make hate speech or elicit harmful responses. However, discussions on sensitive issues can become toxic even if the users are well-intentioned. For safer models in such scenarios, we present the Sensitive Questions and Acceptable Response (SQuARe) dataset, a large-scale Korean dataset of 49k sensitive questions with 42k acceptable and 46k non-acceptable responses. The dataset was constructed leveraging HyperCLOVA in a human-in-the-loop manner based on real news headlines. Experiments show that acceptable response generation significantly improves for HyperCLOVA and GPT-3, demonstrating the efficacy of this dataset.",
}

The potential social harms that large language models pose, such as generating offensive content and reinforcing biases, are steeply rising. Existing works focus on coping with this concern while interacting with ill-intentioned users, such as those who explicitly make hate speech or elicit harmful responses. However, discussions on sensitive issues can become toxic even if the users are well-intentioned. For safer models in such scenarios, we present the Sensitive Questions and Acceptable Response (SQuARe) dataset, a large-scale Korean dataset of 49k sensitive questions with 42k acceptable and 46k non-acceptable responses. The dataset was constructed leveraging HyperCLOVA in a human-in-the-loop manner based on real news headlines. Experiments show that acceptable response generation significantly improves for HyperCLOVA and GPT-3, demonstrating the efficacy of this dataset.
Lee, Hwaran Hong, Seokhee Park, Joonsuk Kim, Takyoung Cha, Meeyoung Choi, Yejin Kim, Byoungpil Kim, Gunhee Lee, Eun-Ju Lim, Yong Oh, Alice Park, Sangchul Ha, Jung-Woo
SQuARe: A Large-Scale Dataset of Sensitive Questions and Acceptable Responses Created through Human-Machine Collaboration