@inproceedings{lee-etal-2023-query,
    title = "Query-Efficient Black-Box Red Teaming via {B}ayesian Optimization",
    author = "Lee, Deokjae  and
      Lee, JunYeong  and
      Ha, Jung-Woo  and
      Kim, Jin-Hwa  and
      Lee, Sang-Woo  and
      Lee, Hwaran  and
      Song, Hyun Oh",
    editor = "Rogers, Anna  and
      Boyd-Graber, Jordan  and
      Okazaki, Naoaki",
    booktitle = "Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)",
    month = jul,
    year = "2023",
    address = "Toronto, Canada",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2023.acl-long.646",
    doi = "10.18653/v1/2023.acl-long.646",
    pages = "11551--11574",
    abstract = "The deployment of large-scale generative models is often restricted by their potential risk of causing harm to users in unpredictable ways. We focus on the problem of black-box red teaming, where a red team generates test cases and interacts with the victim model to discover a diverse set of failures with limited query access. Existing red teaming methods construct test cases based on human supervision or language model (LM) and query all test cases in a brute-force manner without incorporating any information from past evaluations, resulting in a prohibitively large number of queries. To this end, we propose \textit{Bayesian red teaming} (BRT), novel query-efficient black-box red teaming methods based on Bayesian optimization, which iteratively identify diverse positive test cases leading to model failures by utilizing the pre-defined user input pool and the past evaluations. Experimental results on various user input pools demonstrate that our method consistently finds a significantly larger number of diverse positive test cases under the limited query budget than the baseline methods.The source code is available at \url{https://github.com/snu-mllab/Bayesian-Red-Teaming}.",
}

The deployment of large-scale generative models is often restricted by their potential risk of causing harm to users in unpredictable ways. We focus on the problem of black-box red teaming, where a red team generates test cases and interacts with the victim model to discover a diverse set of failures with limited query access. Existing red teaming methods construct test cases based on human supervision or language model (LM) and query all test cases in a brute-force manner without incorporating any information from past evaluations, resulting in a prohibitively large number of queries. To this end, we propose \textit{Bayesian red teaming} (BRT), novel query-efficient black-box red teaming methods based on Bayesian optimization, which iteratively identify diverse positive test cases leading to model failures by utilizing the pre-defined user input pool and the past evaluations. Experimental results on various user input pools demonstrate that our method consistently finds a significantly larger number of diverse positive test cases under the limited query budget than the baseline methods.The source code is available at \url{https://github.com/snu-mllab/Bayesian-Red-Teaming}.
Lee, Deokjae Lee, JunYeong Ha, Jung-Woo Kim, Jin-Hwa Lee, Sang-Woo Lee, Hwaran Song, Hyun Oh
Query-Efficient Black-Box Red Teaming via Bayesian Optimization