Automatic Detection of Toxic South African Tweets Using Support Vector Machines with N-Gram Features

Toxic South African corpus is not available to detect toxic tweets such as offensive, hate, bullying and violent tweets. But there are some offensive and hate speech corpora, mostly in English, which have been used to detect toxic tweets. This paper focuses on automatic detection of toxic South African tweets using a reliable English corpus. The review of text classification models has shown that Support Vector Machines have very often outperformed other classic machine learning algorithms, while word and character n-gram features have performed well with varying prediction performances in different contexts. This paper therefore evaluated the performance of different parameter settings of Support Vector Machines and n-gram features for detection of toxic South African tweets, with a view to hybridize the best among the classifiers. Different combinations of word and character n- gram features were used for the classification. The results show that the Support Vector Machine classifier with set of unigram and bigram as well as set of character n-gram with length sizes from 3 to 7 perform best. By combining the classifiers, the accuracy and F-measure improve from the initial highest Accuracy and F-Measure scores of 0.9085 and 0.94, respectively to 0.9095 and 0.95. The comparison of our results with the performance of previous work on the English corpus shows that our model is reliable.

[1]  Ika Alfina,et al.  Hate speech detection in the Indonesian language: A dataset and preliminary study , 2017, 2017 International Conference on Advanced Computer Science and Information Systems (ICACSIS).

[2]  Ralf Krestel,et al.  Challenges for Toxic Comment Classification: An In-Depth Error Analysis , 2018, ALW.

[3]  Lulamile Mzamo,et al.  Evaluation of the performance of a machine learning lemmatiser for isiXhosa , 2015 .

[4]  Sérgio Nunes,et al.  A Survey on Automatic Detection of Hate Speech in Text , 2018, ACM Comput. Surv..

[5]  Marelie H. Davel,et al.  Language identification of individual words with joint sequence models , 2014, INTERSPEECH.

[6]  Marelie H. Davel,et al.  N-gram based language identification of individual words , 2013 .

[7]  Jie Yang,et al.  Support Vector Machine In Chemistry , 2004 .

[8]  Alan F. Smeaton,et al.  Classifying racist texts using a support vector machine , 2004, SIGIR '04.

[9]  Walter Daelemans,et al.  Multilingual Cross-domain Perspectives on Online Hate Speech , 2018, ArXiv.

[10]  Pete Burnap,et al.  Us and them: identifying cyber hate on Twitter across multiple protected characteristics , 2016, EPJ Data Science.

[11]  Gerrit Reinier Botha Text-based language identification for the South African languages , 2008 .

[12]  Gaël Varoquaux,et al.  Scikit-learn: Machine Learning in Python , 2011, J. Mach. Learn. Res..

[13]  Joel R. Tetreault,et al.  Do Characters Abuse More Than Words? , 2016, SIGDIAL Conference.

[14]  Etienne Barnard,et al.  The NCHLT speech corpus of the South African languages , 2014, SLTU.

[15]  Dirk Hovy,et al.  Hateful Symbols or Hateful People? Predictive Features for Hate Speech Detection on Twitter , 2016, NAACL.

[16]  Yuzhou Wang,et al.  Locate the Hate: Detecting Tweets against Blacks , 2013, AAAI.

[17]  Ingmar Weber,et al.  Automated Hate Speech Detection and the Problem of Offensive Language , 2017, ICWSM.