CSECU_KDE_MA at SemEval-2020 Task 8: A Neural Attention Model for Memotion Analysis

A meme is a pictorial representation of an idea or theme. In the age of emerging volume of social media platforms, memes are spreading rapidly from person to person and becoming a trending ways of opinion expression. However, due to the multimodal characteristics of meme contents, detecting and analyzing the underlying emotion of a meme is a formidable task. In this paper, we present our approach for detecting the emotion of a meme defined in the SemEval-2020 Task 8. Our team CSECU KDE MA employs an attention-based neural network model to tackle the problem. Upon extracting the text contents from a meme using an optical character reader (OCR), we represent it using the distributed representation of words. Next, we perform the convolution based on multiple kernel sizes to obtain the higher-level feature sequences. The feature sequences are then fed into the attentive time-distributed bidirectional LSTM model to learn the long-term dependencies effectively. Experimental results show that our proposed neural model obtained competitive performance among the participants’ systems.

[1]  Kathleen M. Carley,et al.  The evolution of political memes: Detecting and characterizing internet memes with multi-modal deep learning , 2020, Inf. Process. Manag..

[2]  Yuan Yu,et al.  TensorFlow: A system for large-scale machine learning , 2016, OSDI.

[3]  Yoon Kim,et al.  Convolutional Neural Networks for Sentence Classification , 2014, EMNLP.

[4]  Amalia Amalia,et al.  Meme Opinion Categorization by Using Optical Character Recognition (OCR) and Naïve Bayes Algorithm , 2018, 2018 Third International Conference on Informatics and Computing (ICIC).

[5]  John D. Owens,et al.  GPU Computing , 2008, Proceedings of the IEEE.

[6]  Jeffrey Dean,et al.  Distributed Representations of Words and Phrases and their Compositionality , 2013, NIPS.

[7]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[8]  Tanmoy Chakraborty,et al.  Memeify: A Large-Scale Meme Generation System , 2020, COMAD/CODS.

[9]  V AbelL.Peirson,et al.  Dank Learning: Generating Memes Using Deep Neural Networks , 2018, ArXiv.

[10]  Yanqing Zhang,et al.  Visual Sentiment Analysis for Social Images Using Transfer Learning Approach , 2016, 2016 IEEE International Conferences on Big Data and Cloud Computing (BDCloud), Social Computing and Networking (SocialCom), Sustainable Computing and Communications (SustainCom) (BDCloud-SocialCom-SustainCom).

[11]  Tomas Mikolov,et al.  Enriching Word Vectors with Subword Information , 2016, TACL.

[12]  Stephane Fotso,et al.  Attention Fusion Networks: Combining Behavior and E-mail Content to Improve Customer Support , 2018, ArXiv.

[13]  Tanmoy Chakraborty,et al.  SemEval-2020 Task 8: Memotion Analysis- the Visuo-Lingual Metaphor! , 2020, SEMEVAL.

[14]  Amanda Williams,et al.  Racial microaggressions and perceptions of Internet memes , 2016, Comput. Hum. Behav..

[15]  Diyi Yang,et al.  Hierarchical Attention Networks for Document Classification , 2016, NAACL.

[16]  Lukasz Kaiser,et al.  Attention is All you Need , 2017, NIPS.

[17]  Yoshua Bengio,et al.  Neural Machine Translation by Jointly Learning to Align and Translate , 2014, ICLR.

[18]  Ye Zhang,et al.  A Sensitivity Analysis of (and Practitioners’ Guide to) Convolutional Neural Networks for Sentence Classification , 2015, IJCNLP.

[19]  Iyad Rahwan,et al.  Using millions of emoji occurrences to learn any-domain representations for detecting sentiment, emotion and sarcasm , 2017, EMNLP.

[20]  Jin Wang,et al.  Combining Knowledge with Deep Convolutional Neural Networks for Short Text Classification , 2017, IJCAI.

[21]  Masaki Aono,et al.  Visual Sentiment Prediction by Merging Hand-Craft and CNN Features , 2018, 2018 5th International Conference on Advanced Informatics: Concept Theory and Applications (ICAICTA).

[22]  Hugo Gonçalo Oliveira,et al.  One does not simply produce funny memes! - Explorations on the Automatic Generation of Internet humor , 2016, ICCC.

[23]  Ajai Kumar,et al.  Sentiment Extraction from Image-Based Memes Using Natural Language Processing and Machine Learning , 2020 .

[24]  Ickjai Lee,et al.  Document-level multi-topic sentiment classification of Email data with BiLSTM and data augmentation , 2020, Knowl. Based Syst..