Multimodal Hate Speech Detection in Greek Social Media

Hateful and abusive speech presents a major challenge for all online social media platforms. Recent advances in Natural Language Processing and Natural Language Understanding allow for more accurate detection of hate speech in textual streams. This study presents a new multimodal approach to hate speech detection by combining Computer Vision and Natural Language processing models for abusive context detection. Our study focuses on Twitter messages and, more specifically, on hateful, xenophobic, and racist speech in Greek aimed at refugees and migrants. In our approach, we combine transfer learning and fine-tuning of Bidirectional Encoder Representations from Transformers (BERT) and Residual Neural Networks (Resnet). Our contribution includes the development of a new dataset for hate speech classification, consisting of tweet IDs, along with the code to obtain their visual appearance, as they would have been rendered in a web browser. We have also released a pre-trained Language Model trained on Greek tweets, which has been used in our experiments. We report a consistently high level of accuracy (accuracy score = 0.970, f1-score = 0.947 in our best model) in racist and xenophobic speech detection.

[1]  Helen Yannakoudakis,et al.  A Multimodal Framework for the Detection of Hateful Memes , 2020, ArXiv.

[2]  Li Fei-Fei,et al.  ImageNet: A large-scale hierarchical image database , 2009, CVPR.

[3]  Aron Culotta,et al.  Characterizing Variation in Toxic Language by Social Context , 2020, ICWSM.

[4]  Federico Liberatore,et al.  Detecting and Monitoring Hate Speech in Twitter , 2019, Sensors.

[5]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[6]  Zhen Qin,et al.  Are Pre-trained Convolutions Better than Pre-trained Transformers? , 2021, ArXiv.

[7]  Jeffrey Dean,et al.  Distributed Representations of Words and Phrases and their Compositionality , 2013, NIPS.

[8]  Helen Yannakoudakis,et al.  Abusive Language Detection with Graph Convolutional Networks , 2019, NAACL.

[9]  Libby Hemphill,et al.  Quantifying Toxicity and Verbal Violence on Twitter , 2016, CSCW Companion.

[10]  Viviana Patti,et al.  HurtBERT: Incorporating Lexical Features with BERT for the Detection of Abusive Language , 2020, ALW.

[11]  Patrick Pantel,et al.  Preserving integrity in online social networks , 2020, Commun. ACM.

[12]  Lukasz Kaiser,et al.  Attention is All you Need , 2017, NIPS.

[13]  Louis-Philippe Morency,et al.  Multimodal Machine Learning: Integrating Language, Vision and Speech , 2017, ACL.

[14]  Lysandre Debut,et al.  HuggingFace's Transformers: State-of-the-art Natural Language Processing , 2019, ArXiv.

[15]  M. Williams,et al.  Corrigendum to: Hate in the Machine: Anti-Black and Anti-Muslim Social Media Posts as Predictors of Offline Racially and Religiously Aggravated Crime , 2019, The British Journal of Criminology.

[16]  Quoc V. Le,et al.  Distributed Representations of Sentences and Documents , 2014, ICML.

[17]  Viviana Patti,et al.  Resources and benchmark corpora for hate speech detection: a systematic review , 2020, Language Resources and Evaluation.

[18]  Omer Levy,et al.  RoBERTa: A Robustly Optimized BERT Pretraining Approach , 2019, ArXiv.

[19]  Tom De Smedt,et al.  Right-wing German Hate Speech on Twitter: Analysis and Automatic Detection , 2019, ArXiv.

[20]  F. Baider,et al.  Covert hate speech , 2020 .

[21]  Cristina Bosco,et al.  Hate Speech Annotation: Analysis of an Italian Twitter Corpus , 2017, CLiC-it.

[22]  Natalia Gimelshein,et al.  PyTorch: An Imperative Style, High-Performance Deep Learning Library , 2019, NeurIPS.

[23]  Aaron B. Rochlen,et al.  Social media behavior, toxic masculinity, and depression. , 2019, Psychology of Men & Masculinities.

[24]  Ming-Wei Chang,et al.  BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding , 2019, NAACL.

[25]  Ingmar Weber,et al.  Automated Hate Speech Detection and the Problem of Offensive Language , 2017, ICWSM.

[26]  Steven Bethard,et al.  Fine-tuning BERT for multi-domain and multi-label incivil language detection , 2020, ALW.

[27]  Douwe Kiela,et al.  The Hateful Memes Challenge: Detecting Hate Speech in Multimodal Memes , 2020, NeurIPS.

[28]  Panagiotis Karampelas,et al.  Detecting Hate Speech Within the Terrorist Argument: A Greek Case , 2018, 2018 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM).

[29]  Julia Hirschberg,et al.  A Novel Methodology for Developing Automatic Harassment Classifiers for Twitter , 2020, ALW.

[30]  Marcos Zampieri,et al.  Offensive Language Identification in Greek , 2020, LREC.

[31]  Dirk Hovy,et al.  Hateful Symbols or Hateful People? Predictive Features for Hate Speech Detection on Twitter , 2016, NAACL.