SemEval-2020 Task 8: Memotion Analysis- the Visuo-Lingual Metaphor!

Information on social media comprises of various modalities such as textual, visual and audio. NLP and Computer Vision communities often leverage only one prominent modality in isolation to study social media. However, the computational processing of Internet memes needs a hybrid approach. The growing ubiquity of Internet memes on social media platforms such as Facebook, Instagram, and Twiter further suggests that we can not ignore such multimodal content anymore. To the best of our knowledge, there is not much attention towards meme emotion analysis. The objective of this proposal is to bring the attention of the research community towards the automatic processing of Internet memes. The task Memotion analysis released approx 10K annotated memes, with human-annotated labels namely sentiment (positive, negative, neutral), type of emotion (sarcastic, funny, offensive, motivation) and their corresponding intensity. The challenge consisted of three subtasks: sentiment (positive, negative, and neutral) analysis of memes, overall emotion (humour, sarcasm, offensive, and motivational) classification of memes, and classifying intensity of meme emotion. The best performances achieved were F1 (macro average) scores of 0.35, 0.51 and 0.32, respectively for each of the three subtasks.

[1]  Hugo Gonçalo Oliveira,et al.  One does not simply produce funny memes! - Explorations on the Automatic Generation of Internet humor , 2016, ICCC.

[2]  Sergey Ioffe,et al.  Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning , 2016, AAAI.

[3]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[4]  V AbelL.Peirson,et al.  Dank Learning: Generating Memes Using Deep Neural Networks , 2018, ArXiv.

[5]  Arup Baruah,et al.  IIITG-ADBU at SemEval-2020 Task 8: A Multimodal Approach to Detect Offensive, Sarcastic and Humorous Memes , 2020, SEMEVAL.

[6]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[7]  Jeffrey Pennington,et al.  GloVe: Global Vectors for Word Representation , 2014, EMNLP.

[8]  Marek Grzes,et al.  SESAM at SemEval-2020 Task 8: Investigating the Relationship between Image and Text in Sentiment Analysis of Memes , 2020, SEMEVAL.

[9]  Mingxing Xu,et al.  Guoym at SemEval-2020 Task 8: Ensemble-based Classification of Visuo-Lingual Metaphor in Memes , 2020, SEMEVAL.

[10]  Luke S. Zettlemoyer,et al.  Deep Contextualized Word Representations , 2018, NAACL.

[11]  Jonathan Rusert,et al.  NLP_UIOWA at SemEval-2020 Task 8: You’re Not the Only One Cursed with Knowledge - Multi Branch Model Memotion Analysis , 2020, SEMEVAL.

[12]  Amanda Williams,et al.  Racial microaggressions and perceptions of Internet memes , 2016, Comput. Hum. Behav..

[13]  Xuejie Zhang,et al.  YNU-HPCC at SemEval-2020 Task 8: Using a Parallel-Channel Model for Memotion Analysis , 2020, SEMEVAL.

[14]  Enhua Wu,et al.  Squeeze-and-Excitation Networks , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[15]  Louis-Philippe Morency,et al.  M-BERT: Injecting Multimodal Information in the BERT Structure , 2019, ArXiv.

[16]  Siyuan Chen,et al.  UoR at SemEval-2020 Task 8: Gaussian Mixture Modelling (GMM) Based Sampling Approach for Multi-modal Memotion Analysis , 2020, SemEval@COLING.

[17]  Costin-Gabriel Chiru,et al.  UPB at SemEval-2020 Task 8: Joint Textual and Visual Modeling in a Multi-Task Learning Architecture for Memotion Analysis , 2020, SEMEVAL.

[18]  Masaki Aono,et al.  CSECU_KDE_MA at SemEval-2020 Task 8: A Neural Attention Model for Memotion Analysis , 2020, SEMEVAL.

[19]  Himanshu Gupta,et al.  DSC IIT-ISM at SemEval-2020 Task 8: Bi-Fusion Techniques for Deep Meme Emotion Analysis , 2020, SEMEVAL.

[20]  Kazem Taghva,et al.  Using the Google Web 1T 5-Gram Corpus for OCR Error Correction , 2019, 16th International Conference on Information Technology-New Generations (ITNG 2019).

[21]  Andrew Zisserman,et al.  Synthetic Data and Artificial Neural Networks for Natural Scene Text Recognition , 2014, ArXiv.

[22]  Yiming Yang,et al.  XLNet: Generalized Autoregressive Pretraining for Language Understanding , 2019, NeurIPS.

[23]  Li Fei-Fei,et al.  Progressive Neural Architecture Search , 2017, ECCV.

[24]  Indra Budi,et al.  UI at SemEval-2020 Task 8: Text-Image Fusion for Sentiment Classification , 2020, SEMEVAL.

[25]  Arkaitz Zubiaga,et al.  NUAA-QMUL at SemEval-2020 Task 8: Utilizing BERT and DenseNet for Internet Meme Emotion Analysis , 2020, SEMEVAL.

[26]  Ashutosh Modi,et al.  IITK at SemEval-2020 Task 8: Unimodal and Bimodal Sentiment Analysis of Internet Memes , 2020, SEMEVAL.

[27]  Ming-Wei Chang,et al.  BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding , 2019, NAACL.

[28]  Zhen Li,et al.  CN-HIT-MI.T at SemEval-2020 Task 8: Memotion Analysis Based on BERT , 2020, SEMEVAL.

[29]  Anupam Jamatia,et al.  NIT-Agartala-NLP-Team at SemEval-2020 Task 8: Building Multimodal Classifiers to Tackle Internet Humor , 2020, SEMEVAL.

[30]  Els Lefever,et al.  LT3 at SemEval-2020 Task 8: Multi-Modal Multi-Task Learning for Memotion Analysis , 2020, SEMEVAL.

[31]  Noman Islam,et al.  A Survey on Optical Character Recognition System , 2017, ArXiv.

[32]  Preslav Nakov,et al.  SemEval-2019 Task 6: Identifying and Categorizing Offensive Language in Social Media (OffensEval) , 2019, *SEMEVAL.

[33]  Limor Shifman,et al.  “It Gets Better”: Internet memes and the construction of collective identity , 2016, New Media Soc..

[34]  Qingming Huang,et al.  Affective Image Content Analysis: A Comprehensive Survey , 2018, IJCAI.

[35]  Amanpreet Singh,et al.  The Hateful Memes Challenge: Detecting Hate Speech in Multimodal Memes , 2020, NeurIPS.

[36]  Paolo Rosso,et al.  PRHLT-UPV at SemEval-2020 Task 8: Study of Multimodal Techniques for Memes Analysis , 2020, SemEval@COLING.

[37]  Radhika Mamidi,et al.  Gundapusunil at SemEval-2020 Task 8: Multimodal Memotion Analysis , 2020, SEMEVAL.

[38]  Arup Baruah,et al.  KAFK at SemEval-2020 Task 8: Extracting Features from Pre-trained Neural Networks to Classify Internet Memes , 2020, SEMEVAL.

[39]  Dahua Lin,et al.  PolyNet: A Pursuit of Structural Diversity in Very Deep Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[40]  Joanna Isabelle Olszewska,et al.  Active contour based optical character recognition for automated scene understanding , 2015, Neurocomputing.

[41]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[42]  Yiming Yang,et al.  Transformer-XL: Attentive Language Models beyond a Fixed-Length Context , 2019, ACL.

[43]  Ilya Sutskever,et al.  Language Models are Unsupervised Multitask Learners , 2019 .

[44]  Ilanthenral Kandasamy,et al.  Memebusters at SemEval-2020 Task 8: Feature Fusion Model for Sentiment Analysis on Memes Using Transfer Learning , 2020, SEMEVAL.

[45]  Rizwan Ahmed Khan,et al.  Handwritten Optical Character Recognition (OCR): A Comprehensive Systematic Literature Review (SLR) , 2020, IEEE Access.

[46]  Kevin Gimpel,et al.  ALBERT: A Lite BERT for Self-supervised Learning of Language Representations , 2019, ICLR.

[47]  Hiroaki Ozaki,et al.  Hitachi at SemEval-2020 Task 8: Simple but Effective Modality Ensemble for Meme Emotion Recognition , 2020, SEMEVAL.

[48]  Jean H. French.  Image-based memes as sentiment predictors , 2017, 2017 International Conference on Information Society (i-Society).