Hate Me, Hate Me Not: Hate Speech Detection on Facebook

While favouring communications and easing information sharing, Social Network Sites are also used to launch harmful campaigns against specific groups and individuals. Cyberbullism, incitement to self-harm practices, sexual predation are just some of the severe effects of massive online offensives. Moreover, attacks can be carried out against groups of victims and can degenerate in physical violence. In this work, we aim at containing and preventing the alarming diffusion of such hate campaigns. Using Facebook as a benchmark, we consider the textual content of comments appeared on a set of public Italian pages. We first propose a variety of hate categories to distinguish the kind of hate. Crawled comments are then annotated by up to five distinct human annotators, according to the defined taxonomy. Leveraging morpho-syntactical features, sentiment polarity and word embedding lexicons, we design and implement two classifiers for the Italian language, based on different learning algorithms: the first based on Support Vector Machines (SVM) and the second on a particular Recurrent Neural Network named Long Short Term Memory (LSTM). We test these two learning algorithms in order to verify their classification performances on the task of hate speech recognition. The results show the effectiveness of the two classification approaches tested over the first manually annotated Italian Hate Speech Corpus of social media text.

[1]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[2]  Felice Dell ' Orletta Ensemble system for Part-of-Speech tagging , 2009 .

[3]  Preslav Nakov,et al.  SemEval-2016 Task 4: Sentiment Analysis in Twitter , 2016, *SEMEVAL.

[4]  Igor Santos,et al.  Supervised machine learning for the detection of troll profiles in twitter social network: application to a real case of cyberbullying , 2015, Log. J. IGPL.

[5]  Njagi Dennis Gitari,et al.  A Lexicon-based Approach for Hate Speech Detection , 2015, MUE 2015.

[6]  Roberto Di Pietro,et al.  Fame for sale: Efficient detection of fake Twitter followers , 2015, Decis. Support Syst..

[7]  Gene Tsudik,et al.  Exploring Linkability of User Reviews , 2012, ESORICS.

[8]  K. Gwet Handbook of Inter-Rater Reliability: The Definitive Guide to Measuring the Extent of Agreement Among Raters , 2014 .

[9]  Mauro Conti,et al.  FakeBook: Detecting Fake Profiles in On-Line Social Networks , 2012, 2012 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining.

[10]  Lynne Edwards,et al.  ChatCoder: Toward the Tracking and Categorization of Internet Predators , 2009 .

[11]  Huizhi Liang,et al.  UNIMELB at SemEval-2016 Tasks 4A and 4B: An Ensemble of Neural Networks and a Word2Vec Based Model for Sentiment Classification , 2016, *SEMEVAL.

[12]  D. Boyd,et al.  Sociality Through Social Network Sites , 2013 .

[13]  Janyce Wiebe,et al.  Recognizing Contextual Polarity in Phrase-Level Sentiment Analysis , 2005, HLT.

[14]  C. Hardaker,et al.  Trolling in asynchronous computer-mediated communication: From user discussions to academic definitions , 2010 .

[15]  Dolf Trieschnigg,et al.  Improving Cyberbullying Detection with User Context , 2013, ECIR.

[16]  Kuldip K. Paliwal,et al.  Bidirectional recurrent neural networks , 1997, IEEE Trans. Signal Process..

[17]  Bing Liu,et al.  Mining and summarizing customer reviews , 2004, KDD.

[18]  Roberto Di Pietro,et al.  A Criticism to Society (As Seen by Twitter Analytics) , 2014, 2014 IEEE 34th International Conference on Distributed Computing Systems Workshops (ICDCSW).

[19]  Zoubin Ghahramani,et al.  A Theoretically Grounded Application of Dropout in Recurrent Neural Networks , 2015, NIPS.

[20]  Mumit Khan,et al.  Detecting flames and insults in text , 2008 .

[21]  Jing Zhou,et al.  Hate Speech Detection with Comment Embeddings , 2015, WWW.

[22]  Marc Spaniol,et al.  PALADIN: A Pattern Based Approach to Knowledge Discovery in Digital Social Networks , 2006 .

[23]  Ting Liu,et al.  Document Modeling with Gated Recurrent Neural Network for Sentiment Classification , 2015, EMNLP.

[24]  Jure Leskovec,et al.  Antisocial Behavior in Online Discussion Communities , 2015, ICWSM.

[25]  H. S. Chen,et al.  Suicidal Risk Evaluation Using a Similarity-Based Classifier , 2008, ADMA.

[26]  Jeffrey Dean,et al.  Efficient Estimation of Word Representations in Vector Space , 2013, ICLR.

[27]  Peter Kulchyski and , 2015 .

[28]  Vito Pirrelli,et al.  The PAISÀ Corpus of Italian Web Texts , 2014, WaC@EACL.

[29]  E. Cambria,et al.  Do Not Feel The Trolls , 2010 .

[30]  Zhi Xu,et al.  Filtering Offensive Language in Online Communities using Grammatical Relations , 2010 .

[31]  M. Williams,et al.  Hate speech, machine classification and statistical modelling of information flows on Twitter: interpretation and communication for policy decision making , 2014 .

[32]  Carolyn Penstein Rosé,et al.  Detecting offensive tweets via topical feature discovery over a large scale twitter corpus , 2012, CIKM.

[33]  Ellen Spertus,et al.  Smokey: Automatic Recognition of Hostile Messages , 1997, AAAI/IAAI.

[34]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[35]  F. Javier Ortega Detection of dishonest behaviors in on-line networks using graph-based ranking techniques , 2013, AI Commun..

[36]  Malvina Nissim,et al.  Overview of the Evalita 2014 SENTIment POLarity Classification Task , 2014 .