An Expert Annotated Dataset for the Detection of Online Misogyny

Online misogyny is a pernicious social problem that risks making online platforms toxic and unwelcoming to women. We present a new hierarchical taxonomy for online misogyny, as well as an expert labelled dataset to enable automatic classification of misogynistic content. The dataset consists of 6567 labels for Reddit posts and comments. As previous research has found untrained crowdsourced annotators struggle with identifying misogyny, we hired and trained annotators and provided them with robust annotation guidelines. We report baseline classification performance on the binary classification task, achieving accuracy of 0.93 and F1 of 0.43. The codebook and datasets are made freely available for future researchers.

[1]  C. Nau.  Gender hate online: understanding the new anti-feminism , 2021, Feminist Media Studies.

[2]  J. Golbeck.  Online Harassment , 2020, Human Resource Management International Digest.

[3]  Paula Fortuna,et al.  Toxic, Hateful, Offensive or Abusive? What Are We Really Classifying? An Empirical Analysis of Hate Speech Datasets , 2020, LREC.

[4]  Pierangelo Rosati,et al.  Neologising misogyny: Urban Dictionary’s folksonomies of sexual abuse , 2020, New Media Soc..

[5]  Jeremy Blackburn,et al.  From Pick-Up Artists to Incels: A Data-Driven Sketch of the Manosphere , 2020, ArXiv.

[6]  Lluis Gomez,et al.  Exploring Hate Speech Detection in Multimodal Publications , 2019, 2020 IEEE Winter Conference on Applications of Computer Vision (WACV).

[7]  Lysandre Debut,et al.  HuggingFace's Transformers: State-of-the-art Natural Language Processing , 2019, ArXiv.

[8]  Elisabetta Fersini,et al.  Unintended Bias in Misogyny Detection , 2019, 2019 IEEE/WIC/ACM International Conference on Web Intelligence (WI).

[9]  Jing Qian,et al.  A Benchmark Dataset for Learning to Intervene in Online Hate Speech , 2019, EMNLP.

[10]  Patricia Takako Endo,et al.  Data set for automatic detection of online misogynistic speech☆ , 2019, Data in brief.

[11]  Scott A. Hale,et al.  Challenges and frontiers in abusive content detection , 2019, Proceedings of the Third Workshop on Abusive Language Online.

[12]  Yejin Choi,et al.  The Risk of Racial Bias in Hate Speech Detection , 2019, ACL.

[13]  Harith Alani,et al.  Exploring Misogyny across the Manosphere in Reddit , 2019, WebSci.

[14]  David Jurgens,et al.  A Just and Comprehensive Strategy for Using NLP to Address Online Abuse , 2019, ACL.

[15]  Michael Wiegand,et al.  Detection of Abusive Language: the Problem of Biased Datasets , 2019, NAACL.

[16]  Patricia Takako Endo,et al.  A Comparison of Machine Learning Approaches for Detecting Misogynistic Speech in Urban Dictionary , 2019, 2019 International Conference on Cyber Situational Awareness, Data Analytics And Assessment (Cyber SA).

[17]  Emily M. Bender,et al.  Data Statements for Natural Language Processing: Toward Mitigating System Bias and Enabling Better Science , 2018, TACL.

[18]  Donna Zuckerberg,et al.  Not All Dead White Men , 2018 .

[19]  Sérgio Nunes,et al.  A Survey on Automatic Detection of Hate Speech in Text , 2018, ACM Comput. Surv..

[20]  Debbie Ging,et al.  Special issue on online misogyny , 2018, Feminist Media Studies.

[21]  Paolo Rosso,et al.  Automatic Identification and Classification of Misogynistic Language on Twitter , 2018, NLDB.

[22]  Frank Hutter,et al.  Decoupled Weight Decay Regularization , 2017, ICLR.

[23]  Beatríz Villarejo,et al.  Misogyny Online. A Short (and Brutish) History , 2017 .

[24]  Radhika Mamidi,et al.  When does a compliment become sexist? Analysis and classification of ambivalent sexism using twitter data , 2017, NLP+CSS@ACL.

[25]  Ingmar Weber,et al.  Understanding Abuse: A Typology of Abusive Language Detection Subtasks , 2017, ALW@ACL.

[26]  Debbie Ging,et al.  Alphas, Betas, and Incels: Theorizing the Masculinities of the Manosphere , 2017 .

[27]  Michael Wiegand,et al.  A Survey on Hate Speech Detection using Natural Language Processing , 2017, SocialNLP@EACL.

[28]  Ingmar Weber,et al.  Automated Hate Speech Detection and the Problem of Offensive Language , 2017, ICWSM.

[29]  Adrienne Massanari,et al.  #Gamergate and The Fappening: How Reddit’s algorithm, governance, and culture support toxic technocultures , 2017, New Media Soc..

[30]  Bailey Poland,et al.  Haters: Harassment, Abuse, and Violence Online , 2016 .

[31]  Lucas Dixon,et al.  Ex Machina: Personal Attacks Seen at Scale , 2016, WWW.

[32]  Dirk Hovy,et al.  Hateful Symbols or Hateful People? Predictive Features for Hate Speech Detection on Twitter , 2016, NAACL.

[33]  Maria Liakata,et al.  Multi-label Annotation in Scientific Articles - The Multi-label Cancer Risk Assessment Corpus , 2016, LREC.

[34]  Thanassis Tiropanis,et al.  The problem of identifying misogynist language on Twitter (and other online social spaces) , 2016, WebSci.

[35]  E. Jane,et al.  ‘Back to the kitchen, cunt’: speaking the unspeakable about online misogyny , 2014 .

[36]  Trevor Hastie,et al.  Regularization Paths for Generalized Linear Models via Coordinate Descent. , 2010, Journal of statistical software.

[37]  S. Fiske,et al.  Hostile and Benevolent Sexism , 1997 .

[38]  Ming-Wei Chang,et al.  BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding , 2019, NAACL.

[39]  Cristina Bosco,et al.  An Italian Twitter Corpus of Hate Speech against Immigrants , 2018, LREC.

[40]  Paolo Rosso,et al.  Overview of the Task on Automatic Misogyny Identification at IberEval 2018 , 2018, IberEval@SEPLN.

[41]  Maeve Duggan,et al.  Online Harassment 2017 , 2017 .

[42]  Mantilla Karla.  Gendertrolling: Misogyny Adapts to New Media , 2013 .

[43]  Jill Filipovic,et al.  Blogging While Female: How Internet Misogyny Parallels "Real-World" Harassment , 2007 .