@inproceedings{zhou-etal-2023-cobra,
    title = "{COBRA} Frames: Contextual Reasoning about Effects and Harms of Offensive Statements",
    author = "Zhou, Xuhui  and
      Zhu, Hao  and
      Yerukola, Akhila  and
      Davidson, Thomas  and
      Hwang, Jena D.  and
      Swayamdipta, Swabha  and
      Sap, Maarten",
    editor = "Rogers, Anna  and
      Boyd-Graber, Jordan  and
      Okazaki, Naoaki",
    booktitle = "Findings of the Association for Computational Linguistics: ACL 2023",
    month = jul,
    year = "2023",
    address = "Toronto, Canada",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2023.findings-acl.392",
    doi = "10.18653/v1/2023.findings-acl.392",
    pages = "6294--6315",
    abstract = "Warning: This paper contains content that may be offensive or upsetting. Understanding the harms and offensiveness of statements requires reasoning about the social and situational context in which statements are made. For example, the utterance {``}your English is very good{''} may implicitly signal an insult when uttered by a white man to a non-white colleague, but uttered by an ESL teacher to their student would be interpreted as a genuine compliment. Such contextual factors have been largely ignored by previous approaches to toxic language detection. We introduce COBRA frames, the first context-aware formalism for explaining the intents, reactions, and harms of offensive or biased statements grounded in their social and situational context. We create COBRACORPUS, a dataset of 33k potentially offensive statements paired with machine-generated contexts and free-text explanations of offensiveness, implied biases, speaker intents, and listener reactions. To study the contextual dynamics of offensiveness, we train models to generate COBRA explanations, with and without access to the context. We find that explanations by context-agnostic models are significantly worse than by context-aware ones, especially in situations where the context inverts the statement{'}s offensiveness (29{\%} accuracy drop). Our work highlights the importance and feasibility of contextualized NLP by modeling social factors.",
}

Warning: This paper contains content that may be offensive or upsetting. Understanding the harms and offensiveness of statements requires reasoning about the social and situational context in which statements are made. For example, the utterance {``}your English is very good{''} may implicitly signal an insult when uttered by a white man to a non-white colleague, but uttered by an ESL teacher to their student would be interpreted as a genuine compliment. Such contextual factors have been largely ignored by previous approaches to toxic language detection. We introduce COBRA frames, the first context-aware formalism for explaining the intents, reactions, and harms of offensive or biased statements grounded in their social and situational context. We create COBRACORPUS, a dataset of 33k potentially offensive statements paired with machine-generated contexts and free-text explanations of offensiveness, implied biases, speaker intents, and listener reactions. To study the contextual dynamics of offensiveness, we train models to generate COBRA explanations, with and without access to the context. We find that explanations by context-agnostic models are significantly worse than by context-aware ones, especially in situations where the context inverts the statement{'}s offensiveness (29{\%} accuracy drop). Our work highlights the importance and feasibility of contextualized NLP by modeling social factors.
Zhou, Xuhui Zhu, Hao Yerukola, Akhila Davidson, Thomas Hwang, Jena D. Swayamdipta, Swabha Sap, Maarten
COBRA Frames: Contextual Reasoning about Effects and Harms of Offensive Statements