Overview of the Evalita 2018 Task on Automatic Misogyny Identification (AMI)

English. Automatic Misogyny Identification (AMI) is a new shared task proposed for the first time at the Evalita 2018 evaluation campaign. The AMI challenge, based on both Italian and English tweets, is distinguished into two subtasks, i.e. Subtask A on misogyny identification and Subtask B about misogynistic behaviour categorization and target classification. Regarding the Italian language, we have received a total of 13 runs for Subtask A and 11 runs for Subtask B. Concerning the English language, we received 26 submissions for Subtask A and 23 runs for Subtask B. The participating systems have been distinguished according to the language, counting 6 teams for Italian and 10 teams for English. We present here an overview of the AMI shared task, the datasets, the evaluation methodology, the results obtained by the participants and a discussion of the methodology adopted by the teams. Finally, we draw some conclusions and discuss future work. Italiano. Automatic Misogyny Identification (AMI) è un nuovo shared task proposto per la prima volta nella campagna di valutazione Evalita 2018. La sfida AMI, basata su tweet italiani e inglesi, si distingue in due sottotask ossia Subtask A relativo al riconoscimento della misoginia e Subtask B relativo alla categorizzazione di espressioni misogine e alla classificazione del soggetto target. Per quanto riguarda la lingua italiana, sono stati ricevuti un totale di 13 run per il Subtask A e 11 run per il Subtask B. Per quanto riguarda la lingua inglese, sono stati ricevuti 26 run per il Subtask A e 23 per Subtask B. I sistemi partecipanti sono stati distinti in base alla lingua, raccogliendo un totale di 6 team partecipanti per l’italiano e 10 team per l’inglese. Presentiamo di seguito una sintesi dello shared task AMI, i dataset, la metodologia di valutazione, i risultati ottenuti dai partecipanti e una discussione sulle metodologie adottate dai diversi team. Infine, vengono discusse conclusioni e delineati gli sviluppi futuri.

[1]  Viviana Patti,et al.  Automatic Identification of Misogyny in English and Italian Tweets at EVALITA 2018 with a Multilingual Hate Lexicon , 2018, EVALITA@CLiC-it.

[2]  Paolo Rosso,et al.  Overview of the Task on Automatic Misogyny Identification at IberEval 2018 , 2018, IberEval@SEPLN.

[3]  Davide Buscaldi.  Tweetaneuse @ AMI EVALITA2018: Character-based Models for the Automatic Misogyny Identification Task (Short Paper) , 2018, EVALITA@CLiC-it.

[4]  Andrea Esuli,et al.  SentiWordNet 3.0: An Enhanced Lexical Resource for Sentiment Analysis and Opinion Mining , 2010, LREC.

[5]  John Cardiff,et al.  Misogyny Detection and Classification in English Tweets: The Experience of the ITT Team , 2018, EVALITA@CLiC-it.

[6]  J. Golbeck.  Online Harassment , 2020, Human Resource Management International Digest.

[7]  Thanassis Tiropanis,et al.  The problem of identifying misogynist language on Twitter (and other online social spaces) , 2016, WebSci.

[8]  Nicole Novielli,et al.  UNIBA at EVALITA 2014-SENTIPOLC Task: Predicting tweet sentiment polarity combining micro-blogging, lexicon and semantic features , 2014 .

[9]  Tommaso Caselli,et al.  Evalita 2018: Overview on the 6th Evaluation Campaign of Natural Language Processing and Speech Tools for Italian , 2018, EVALITA@CLiC-it.

[10]  Malvina Nissim,et al.  Bleaching Text: Abstract Features for Cross-lingual Gender Prediction , 2018, ACL.

[11]  Amir Bakarov,et al.  Vector Space Models for Automatic Misogyny Identification , 2018 .

[12]  Manuel Montes-y-Gómez,et al.  Exploration of Misogyny in Spanish and English Tweets , 2018, IberEval@SEPLN.

[13]  Bailey Poland,et al.  Haters: Harassment, Abuse, and Violence Online , 2016 .

[14]  Paolo Rosso,et al.  Automatic Identification and Classification of Misogynistic Language on Twitter , 2018, NLDB.

[15]  Andrea Carnaghi,et al.  Social acceptability of sexist derogatory and sexist objectifying slurs across contexts , 2015 .

[16]  Martine De Cock,et al.  Detecting Hate Speech Against Women in English Tweets , 2018, EVALITA@CLiC-it.

[17]  Jeffrey Pennington,et al.  GloVe: Global Vectors for Word Representation , 2014, EMNLP.