YNU-HPCC at SemEval-2020 Task 8: Using a Parallel-Channel Model for Memotion Analysis

In recent years, the growing ubiquity of Internet memes on social media platforms, such as Facebook, Instagram, and Twitter, has become a topic of immense interest. However, the classification and recognition of memes is much more complicated than that of social text since it involves visual cues and language understanding. To address this issue, this paper proposed a parallel-channel model to process the textual and visual information in memes and then analyze the sentiment polarity of memes. In the shared task of identifying and categorizing memes, we preprocess the dataset according to the language behaviors on social media. Then, we adapt and fine-tune the Bidirectional Encoder Representations from Transformers (BERT), and two types of convolutional neural network models (CNNs) were used to extract the features from the pictures. We applied an ensemble model that combined the BiLSTM, BIGRU, and Attention models to perform cross domain suggestion mining. The officially released results show that our system performs better than the baseline algorithm. Our team won nineteenth place in subtask A (Sentiment Classification). The code of this paper is availabled at : this https URL.

[1]  Yoshua Bengio,et al.  Neural Machine Translation by Jointly Learning to Align and Translate , 2014, ICLR.

[2]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[3]  Lukasz Kaiser,et al.  Attention is All you Need , 2017, NIPS.

[4]  Wei Chen,et al.  A Character-Aware Encoder for Neural Machine Translation , 2016, COLING.

[5]  Rongrong Ji,et al.  Large-scale visual sentiment ontology and detectors using adjective noun pairs , 2013, ACM Multimedia.

[6]  Jiebo Luo,et al.  Progressive Self-Supervised Attention Learning for Aspect-Level Sentiment Analysis , 2019, ACL.

[7]  Jürgen Schmidhuber,et al.  LSTM: A Search Space Odyssey , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[8]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[9]  Yoshua Bengio,et al.  Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation , 2014, EMNLP.

[10]  Amalia Amalia,et al.  Meme Opinion Categorization by Using Optical Character Recognition (OCR) and Naïve Bayes Algorithm , 2018, 2018 Third International Conference on Informatics and Computing (ICIC).

[11]  K. Robert Lai,et al.  Investigating Dynamic Routing in Tree-Structured LSTM for Sentiment Analysis , 2019, EMNLP.

[12]  Maofu Liu,et al.  An image-text consistency driven multimodal sentiment analysis approach for social media , 2019, Inf. Process. Manag..

[13]  Seth Flaxman,et al.  Multimodal Sentiment Analysis To Explore the Structure of Emotions , 2018, KDD.

[14]  Gaël Varoquaux,et al.  Scikit-learn: Machine Learning in Python , 2011, J. Mach. Learn. Res..

[15]  Liang-Chih Yu,et al.  Tree-Structured Regional CNN-LSTM Model for Dimensional Sentiment Analysis , 2020, IEEE/ACM Transactions on Audio, Speech, and Language Processing.