Memotion 2: Dataset on Sentiment and Emotion Analysis of Memes (short paper)

Memes are commonly used in social media platforms for humour. Generally, memes consist of an image and embedded text. Memes can be used to spread hate or misinformation, hence it is important to study them. The Memotion task [1] conducted at SemEval 2020, released a data of 10k memes annotated with sentiment label (task A), emotion label (task B) and emotion intensity label (task C). It received ≈ 30 run submissions and 27 papers. However, the best f1 scores were only 0.35, 0.51 and 0.32 respectively for task A, task B, and task C, which shows the need for more extensive research on this topic. In this paper, we release a new dataset, Memotion 2 which has 10k annotated memes along the same directions as Memotion 1.0 This paper detailed baseline system on the Memotion 2.0 data.

[1]  Tanmoy Chakraborty,et al.  SemEval-2020 Task 9: Overview of Sentiment Analysis of Code-Mixed Tweets , 2020, SEMEVAL.

[2]  Mihael Arcan,et al.  Multimodal Meme Dataset (MultiOFF) for Identifying Offensive Content in Image and Text , 2020, TRAC.

[3]  Douwe Kiela,et al.  The Hateful Memes Challenge: Detecting Hate Speech in Multimodal Memes , 2020, NeurIPS.

[4]  Scott Anderson.  Scott , 2020 .

[5]  Lluis Gomez,et al.  Exploring Hate Speech Detection in Multimodal Publications , 2019, 2020 IEEE Winter Conference on Applications of Computer Vision (WACV).

[6]  Jing Qian,et al.  A Benchmark Dataset for Learning to Intervene in Online Hate Speech , 2019, EMNLP.

[7]  Yejin Choi,et al.  The Risk of Racial Bias in Hate Speech Detection , 2019, ACL.

[8]  Ingmar Weber,et al.  Racial Bias in Hate Speech and Abusive Language Detection Datasets , 2019, Proceedings of the Third Workshop on Abusive Language Online.

[9]  Seth Flaxman,et al.  Multimodal Sentiment Analysis To Explore the Structure of Emotions , 2018, KDD.

[10]  Ingmar Weber,et al.  Automated Hate Speech Detection and the Problem of Offensive Language , 2017, ICWSM.

[11]  Zeerak Waseem,et al.  Are You a Racist or Am I Seeing Things? Annotator Influence on Hate Speech Detection on Twitter , 2016, NLP+CSS@EMNLP.

[12]  Amanda Williams,et al.  Racial microaggressions and perceptions of Internet memes , 2016, Comput. Hum. Behav..

[13]  Limor Shifman,et al.  “It Gets Better”: Internet memes and the construction of collective identity , 2016, New Media Soc..

[14]  Dirk Hovy,et al.  Hateful Symbols or Hateful People? Predictive Features for Hate Speech Detection on Twitter , 2016, NAACL.

[15]  Pete Burnap,et al.  Us and them: identifying cyber hate on Twitter across multiple protected characteristics , 2016, EPJ Data Science.

[16]  Shivakant Mishra,et al.  Prediction of Cyberbullying Incidents on the Instagram Social Network , 2015, ArXiv.

[17]  Rada Mihalcea,et al.  Towards multimodal sentiment analysis: harvesting opinions from the web , 2011, ICMI '11.

[18]  Christopher Potts,et al.  Learning Word Vectors for Sentiment Analysis , 2011, ACL.

[19]  Michael Wiegand,et al.  Overview of GermEval Task 2, 2019 Shared Task on the Identification of Offensive Language , 2019, KONVENS.