Emojis as anchors to detect Arabic offensive language and hate speech

We introduce a generic, language-independent method to collect a large percentage of offensive and hate tweets regardless of their topics or genres. We harness the extralinguistic information embedded in the emojis to collect a large number of offensive tweets. We apply the proposed method on Arabic tweets and compare it with English tweets—analyzing key cultural differences. We observed a constant usage of these emojis to represent offensiveness throughout different timespans on Twitter. We manually annotate and publicly release the largest Arabic dataset for offensive, fine-grained hate speech, vulgar, and violence content. Furthermore, we benchmark the dataset for detecting offensiveness and hate speech using different transformer architectures and perform in-depth linguistic analysis. We evaluate our models on external datasets—a Twitter dataset collected using a completely different method, and a multi-platform dataset containing comments from Twitter, YouTube, and Facebook, for assessing generalization capability. Competitive results on these datasets suggest that the data collected using our method capture universal characteristics of offensive language. Our findings also highlight the common words used in offensive communications, common targets for hate speech, specific patterns in violence tweets, and pinpoint common classification errors that can be attributed to limitations of NLP models. We observe that even state-of-the-art transformer models may fail to take into account culture, background, and context or understand nuances present in real-world data such as sarcasm.

[1]  Firoj Alam,et al.  Detecting Propaganda Techniques in Memes , 2021, ACL.

[2]  Kareem Darwish,et al.  Cross-lingual Emotion Detection , 2021, LREC.

[3]  Ahmed Abdelali,et al.  Pre-Training BERT on Arabic Tweets: Practical Considerations , 2021, ArXiv.

[4]  Preslav Nakov,et al.  SemEval-2020 Task 12: Multilingual Offensive Language Identification in Social Media (OffensEval 2020) , 2020, SEMEVAL.

[5]  Ahmed Abdelali,et al.  QADI: Arabic Dialect Identification in the Wild , 2020, WANLP.

[6]  Douwe Kiela,et al.  The Hateful Memes Challenge: Detecting Hate Speech in Multimodal Memes , 2020, NeurIPS.

[7]  Bernard J. Jansen,et al.  A Multi-Platform Arabic News Comment Dataset for Offensive Language Detection , 2020, LREC.

[8]  Walid Magdy,et al.  Overview of OSACT4 Arabic Offensive Language Detection Shared Task , 2020, OSACT.

[9]  Fatemah Husain,et al.  OSACT4 Shared Task on Offensive Language Detection: Intensive Preprocessing-Based Approach , 2020, OSACT.

[10]  Hamdy Mubarak,et al.  Arabic Offensive Language on Twitter: Analysis and Experiments , 2020, WANLP.

[11]  Domenico Talia,et al.  Learning Political Polarization on Social Media Using Neural Networks , 2020, IEEE Access.

[12]  Hazem M. Hajj,et al.  AraBERT: Transformer-based Model for Arabic Language Understanding , 2020, OSACT.

[13]  Bernard J. Jansen,et al.  Developing an online hate classifier for multiple social media platforms , 2020, Human-centric Computing and Information Sciences.

[14]  Myle Ott,et al.  Unsupervised Cross-lingual Representation Learning at Scale , 2019, ACL.

[15]  Yangqiu Song,et al.  Multilingual and Multi-Aspect Hate Speech Analysis , 2019, EMNLP.

[16]  Marco Guerini,et al.  CONAN - COunter NArratives through Nichesourcing: a Multilingual Dataset of Responses to Fight Online Hate Speech , 2019, ACL.

[17]  Qiaozhu Mei,et al.  Decoding the New World Language: Analyzing the Popularity, Roles, and Utility of Emojis , 2019, WWW.

[18]  Shivakant Mishra,et al.  International Conference on Advances in Social Networks Analysis and Mining ( ASONAM ) Are They Our Brothers ? Analysis and Detection of Religious Hate Speech in the Arabic Twittersphere , 2018 .

[19]  Patrizia Paggio,et al.  Investigating Redundancy in Emoji Use: Study on a Twitter Based Corpus , 2017, WASSA@EMNLP.

[20]  Walid Magdy,et al.  Abusive Language Detection on Arabic Social Media , 2017, ALW@ACL.

[21]  Preslav Nakov,et al.  Seminar Users in the Arabic Twitter Sphere , 2017, SocInfo.

[22]  Ingmar Weber,et al.  Automated Hate Speech Detection and the Problem of Offensive Language , 2017, ICWSM.

[23]  Ploypailin Intapong,et al.  Assessing symptoms of excessive SNS usage based on user behavior and emotion , 2016, HCI.

[24]  Marco Tulio Ribeiro,et al.  "Why Should I Trust You?": Explaining the Predictions of Any Classifier , 2016, HLT-NAACL Demos.

[25]  Xue Liu,et al.  ISC: An Iterative Social Based Classifier for Adult Account Detection on Twitter , 2015, IEEE Transactions on Knowledge and Data Engineering.

[26]  Kareem Darwish,et al.  Using Twitter to Collect a Multi-Dialectal Corpus of Arabic , 2014, ANLP@EMNLP.

[27]  J. Waldron,et al.  The Harm in Hate Speech , 2012 .

[28]  Jacob Ratkiewicz,et al.  Political Polarization on Twitter , 2011, ICWSM.

[29]  J. Platt Sequential Minimal Optimization : A Fast Algorithm for Training Support Vector Machines , 1998 .

[30]  J. R. Landis,et al.  The measurement of observer agreement for categorical data. , 1977, Biometrics.

[31]  Hamdy Mubarak,et al.  Adult Content Detection on Arabic Twitter: Analysis and Experiments , 2021, WANLP.

[32]  Michael Wiegand,et al.  Exploiting Emojis for Abusive Language Detection , 2021, EACL.

[33]  Ahmed Abdelali,et al.  ASAD: Arabic Social media Analytics and unDerstanding , 2021, EACL.

[34]  Isabelle Augenstein,et al.  Detecting Abusive Language on Online Platforms: A Critical Analysis , 2021, ArXiv.

[35]  Abdessamad Benlahbib,et al.  LISAC FSDM-USMBA Team at SemEval-2020 Task 12: Overcoming AraBERT’s pretrain-finetune discrepancy for Arabic offensive language identification , 2020, SEMEVAL.

[36]  S. A. Chowdhury,et al.  Improving Arabic Text Categorization Using Transformer Training Diversification , 2020, WANLP.

[37]  Saleh Alhazbi,et al.  Behavior-Based Machine Learning Approaches to Identify State-Sponsored Trolls on Twitter , 2020, IEEE Access.

[38]  Kareem Darwish,et al.  Spam Detection on Arabic Twitter , 2020, SocInfo.

[39]  Ahmed Abdelali,et al.  ALT at SemEval-2020 Task 12: Arabic and English Offensive Language Identification in Social Media , 2020, SEMEVAL.

[40]  Raghad Alshaalan,et al.  Hate Speech Detection in Saudi Twittersphere: A Deep Learning Approach , 2020, WANLP.

[41]  Ming-Wei Chang,et al.  BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding , 2019, NAACL.

[42]  Giovanni Semeraro,et al.  AlBERTo: Italian BERT Language Understanding Model for NLP Challenging Tasks Based on Tweets , 2019, CLiC-it.

[43]  Hassan Alhuzali,et al.  Think Before Your Click: Data and Models for Adult Content in Arabic Twitter , 2018 .

[44]  Fikret Gülaçti,et al.  The effect of perceived social support on subjective well-being , 2010 .