Offensive Comments in the Brazilian Web: a dataset and baseline results

Brazilian Web users are among the most active in social networks and very keen on interacting with others. Offensive comments, known as hate speech, have been plaguing online media and originating a number of lawsuits against companies which publish Web content. Given the massive number of user generated text published on a daily basis, manually filtering offensive comments becomes infeasible. The identification of offensive comments can be treated as a supervised classification task. In order to obtain a model to classify comments, an annotated dataset containing positive and negative examples is necessary. The lack of such a dataset in Portuguese, limits the development of detection approaches for this language. In this paper, we describe how we created annotated datasets of offensive comments for Portuguese by collecting news comments on the Brazilian Web. In addition, we provide classification results achieved by standard classification algorithms on these datasets which can serve as baseline for future work on this topic.

[1]  Ying Chen,et al.  Detecting Offensive Language in Social Media to Protect Adolescent Online Safety , 2012, 2012 International Conference on Privacy, Security, Risk and Trust and 2012 International Confernece on Social Computing.

[2]  I-Hsien Ting,et al.  Content matters: A study of hate groups detection based on social networks analysis and web mining , 2013, 2013 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM 2013).

[3]  Lucas Dixon,et al.  Ex Machina: Personal Attacks Seen at Scale , 2016, WWW.

[4]  Yuzhou Wang,et al.  Locate the Hate: Detecting Tweets against Blacks , 2013, AAAI.

[5]  Shuhua Liu,et al.  Text Classification Models for Web Content Filtering and Online Safety , 2015, 2015 IEEE International Conference on Data Mining Workshop (ICDMW).

[6]  Fabrício Benevenuto,et al.  Analyzing the Targets of Hate in Online Social Media , 2016, ICWSM.

[7]  J. Fleiss.  Measuring nominal scale agreement among many raters. , 1971 .

[8]  Julia Hirschberg,et al.  Detecting Hate Speech on the World Wide Web , 2012 .

[9]  Jennifer Jie Xu,et al.  Mining communities and their relationships in blogs: A study of online hate groups , 2007, Int. J. Hum. Comput. Stud..

[10]  Carolyn Penstein Rosé,et al.  Detecting offensive tweets via topical feature discovery over a large scale twitter corpus , 2012, CIKM.

[11]  Jing Zhou,et al.  Hate Speech Detection with Comment Embeddings , 2015, WWW.

[12]  Joel R. Tetreault,et al.  Abusive Language Detection in Online User Content , 2016, WWW.

[13]  Björn Ross,et al.  Measuring the Reliability of Hate Speech Annotations: The Case of the European Refugee Crisis , 2016, ArXiv.

[14]  Kevin W. Saunders.  What about Hate Speech , 2011 .

[15]  Zeerak Waseem,et al.  Are You a Racist or Am I Seeing Things? Annotator Influence on Hate Speech Detection on Twitter , 2016, NLP+CSS@EMNLP.

[16]  Elizabeth F. Churchill,et al.  Using Crowdsourcing to Improve Profanity Detection , 2012, AAAI Spring Symposium: Wisdom of the Crowd.

[17]  Stan Matwin,et al.  Offensive Language Detection Using Multi-level Classification , 2010, Canadian Conference on AI.