offensive language
84843035
SemEval 2019 Task 6: Identifying and Categorizing Offensive Language in Social Media (OffensEval)
We present the results and the main findings of SemEval 2019 Task 6 on Identifying and Categorizing Offensive Language in Social Media (OffensEval) The task was based on a new dataset, the Offensive Language Identification Dataset (OLID) which contains over 14,000 English tweets, and it featured three sub tasks. In sub task A, systems were asked to discriminate between offensive and non offensive posts. In sub task B, systems had to identify the type of offensive content in the post. Finally, in sub task C, systems had to detect the target of the offensive posts. OffensEval attracted a large number of participants and it was one of the most popular tasks in SemEval 2019. In total, nearly 800 teams signed up to participate in the task and 115 of them submitted results, which are presented and analyzed in this report.
*SEMEVAL
Marcos Zampieri Shervin Malmasi Preslav Nakov Sara Rosenthal Noura Farra Ritesh Kumar
1733167
Automated Hate Speech Detection and the Problem of Offensive Language
A key challenge for automatic hate speech detection on social media is the separation of hate speech from other instances of offensive language. Lexical detection methods tend to have low precision because they classify all messages containing particular terms as hate speech and previous work using supervised learning has failed to distinguish between the two categories. We used a crowd sourced hate speech lexicon to collect tweets containing hate speech keywords. We use crowd sourcing to label a sample of these tweets into three categories: those containing hate speech, only offensive language, and those with neither. We train a multi class classifier to distinguish between these different categories. Close analysis of the predictions and the errors shows when we can reliably separate hate speech from other offensive language and when this differentiation is more difficult. We find that racist and homophobic tweets are more likely to be classified as hate speech but that sexist tweets are generally classified as offensive. Tweets without explicit hate keywords are also more difficult to classify.
ICWSM
Thomas Davidson Dana Warmsley Michael W Macy Ingmar Weber
207917760
Overview of the GermEval 2018 Shared Task on the Identification of Offensive Language
We present the pilot edition of the GermEval Shared Task on the Identification of Offensive Language. This shared task deals with the classification of German tweets from Twitter. It comprises two tasks, a coarse grained binary classification task and a fine grained multi class classification task. The shared task had 20 participants submitting 51 runs for the coarse grained task and 25 runs for the fine grained task. Since this is a pilot task, we describe the process of extracting the raw data for the data collection and the annotation schema. We evaluate the results of the systems submitted to the shared task. The shared task homepage can be found at https:/projects.cai. fbi.h da.de/iggsa/
Michael Wiegand Melanie Siegel
12985409
Detecting Offensive Language in Tweets Using Deep Learning
This paper addresses the important problem of discerning hateful content in social media. We propose a detection scheme that is an ensemble of Recurrent Neural Network (RNN) classifiers, and it incorporates various features associated with userrelated information, such as the users' tendency towards racism or sexism. These data are fed as input to the above classifiers along with the word frequency vectors derived from the textual content. Our approach has been evaluated on a publicly available corpus of 16k tweets, and the results demonstrate its effectiveness in comparison to existing state of the art solutions. More specifically, our scheme can successfully distinguish racism and sexism messages from normal text, and achieve higher classification quality than current state of the art algorithms.
ArXiv
Georgios K Pitsilis Heri Ramampiaro Helge Langseth
148567611
Offensive Language Detection with Neural Networks for Germeval Task 2018
In this paper we describe our submissions to task I of the GermEval 2018 Shared Task with the goal of identifying offensive language in a set of German tweets. We experiment with two neural architectures and different features. Our submission consists of 3 runs using ensembles of different neural network architectures, each achieving approximately 78 macro F1 measure on the last 500 tweets from the training set. The source code for our experiments is publicly available on Github. 1
Dominik Stammbach
29165442
Fighting Offensive Language on Social Media with Unsupervised Text Style Transfer
We introduce a new approach to tackle the problem of offensive language in online social media. Our approach uses unsupervised text style transfer to translate offensive sentences into non offensive ones. We propose a new method for training encoder decoders using non parallel data that combines a collaborative classifier, attention and the cycle consistency loss. Experimental results on data from Twitter and Reddit show that our method outperforms a state of the art text style transfer system in two out of three quantitative metrics and produces reliable non offensive transferred sentences.
ACL
Cicero Nogueira dos Santos Igor V Melnyk Inkit Padhi
214802624
Arabic Offensive Language on Twitter: Analysis and Experiments
Detecting offensive language on Twitter has many applications ranging from detecting/predicting bullying to measuring polarization. In this paper, we focus on building a large Arabic offensive tweet dataset. We introduce a method for building a dataset that is not biased by topic, dialect, or target. We produce the largest Arabic dataset to date with special tags for vulgarity and hate speech. We thoroughly analyze the dataset to determine which topics, dialects, and gender are most associated with offensive tweets and how Arabic speakers useoffensive language. Lastly, we conduct many experiments to produce strong results (F1 =83.2) on the dataset using SOTA techniques.
WANLP
Hamdy Mubarak Ammar Rashed Kareem Darwish Younes Samih Ahmed Abdelali
212676302
Towards Automatic Detection and Explanation of Hate Speech and Offensive Language
The use of hate speech and offensive language online has become widely recognized as a critical social problem plaguing today's Internet users. Previous research in the detection of hate speech and offensive language has primarily focused on using machine learning approaches to naively detect hate speech and offensive language, without explaining the reasons for their detection. In this work, we introduce a novel hate speech and offensive language defense system called HateDefender, which consists of a detection model based on deep Long Short term Memory (LSTM) neural networks and an explanation model based on the gating signals of LSTMs. HateDefender effectively detects hate speech and offensive language (average accuracy of 90.82% and 89.10% on hate speech and offensive language, respectively) and explains their factors by pinpointing the exact words that are responsible for causing them. Our system uses these explanations for the effective intervention of such incidents online.
IWSPA@CODASPY
Wyatt Dorris Ruijia (Roger) Hu Nishant Vishwamitra Feng Luo Matthew Costello
198187869
FuzzE: Fuzzy Fairness Evaluation of Offensive Language Classifiers on African American English
Hate speech and offensive language are rampant on social media. Machine learning has provided a way to moderate foul language at scale. However, much of the current research focuses on overall performance. Models may perform poorly on text written in a minority dialectal language. For instance, a hate speech classifier may produce more false positives on tweets written in African American Vernacular English (AAVE) To measure these problems, we need text written in both AAVE and Standard American English (SAE) Unfortunately, it is challenging to curate data for all linguistic styles in a timely manner especially when we are constrained to specific problems, social media platforms, or by limited resources. In this paper, we answer the question, "How can we evaluate the performance of classifiers across minority dialectal languages when they are not present within a particular dataset?" Specifically, we propose an automated fairness fuzzing tool called FuzzE to quantify the fairness of text classifiers applied to AAVE text using a dataset that only contains text written in SAE. Overall, we find that the fairness estimates returned by our technique moderately correlates with the use of real ground truth AAVE text. Warning: Offensive language is displayed in this manuscript.
AAAI
Anthony Rios
210116806
Offensive Language Detection: A Comparative Analysis
Offensive behaviour has become pervasive in the Internet community. Individuals take the advantage of anonymity in the cyber world and indulge in offensive communications which they may not consider in the real life. Governments, online communities, companies etc are investing into prevention of offensive behaviour content in social media. One of the most effective solution for tacking this enigmatic problem is the use of computational techniques to identify offensive content and take action. The current work focuses on detecting offensive language in English tweets. The dataset used for the experiment is obtained from SemEval 2019 Task 6 on Identifying and Categorizing Offensive Language in Social Media (OffensEval) The dataset contains 14,460 annotated English tweets. The present paper provides a comparative analysis and Random kitchen sink (RKS) based approach for offensive language detection. We explore the effectiveness of Google sentence encoder, Fasttext, Dynamic mode decomposition (DMD) based features and Random kitchen sink (RKS) method for offensive language detection. From the experiments and evaluation we observed that RKS with fastetxt achieved competing results. The evaluation measures used are accuracy, precision, recall, f1 score.
ArXiv
T VyshnavM S Sachin Kumar P SomanK