AOMD: An Analogy-aware Approach to Offensive Meme Detection on Social Media

This paper focuses on an important problem of detecting offensive analogy meme on online social media where the visual content and the texts/captions of the meme together make an analogy to convey the offensive information. Existing offensive meme detection solutions often ignore the implicit relation between the visual and textual contents of the meme and are insufficient to identify the offensive analogy memes. Two important challenges exist in accurately detecting the offensive analogy memes: i) it is not trivial to capture the analogy that is often implicitly conveyed by a meme; ii) it is also challenging to effectively align the complex analogy across different data modalities in a meme. To address the above challenges, we develop a deep learning based Analogy-aware Offensive Meme Detection (AOMD) framework to learn the implicit analogy from the multi-modal contents of the meme and effectively detect offensive analogy memes. We evaluate AOMD on two real-world datasets from online social media. Evaluation results show that AOMD achieves significant performance gains compared to state-of-the-art baselines by detecting offensive analogy memes more accurately.

[1]  Xavier Giró-i-Nieto,et al.  Hate Speech in Pixels: Detection of Offensive Memes towards Automatic Moderation , 2019, ArXiv.

[2]  Michael S. Bernstein,et al.  Anyone Can Become a Troll: Causes of Trolling Behavior in Online Discussions , 2017, CSCW.

[3]  Graham W. Taylor,et al.  Deep Multimodal Learning: A Survey on Recent Advances and Trends , 2017, IEEE Signal Processing Magazine.

[4]  Jewgeni Rose,et al.  Detecting Hate Speech in Memes Using Multimodal Deep Learning Approaches: Prize-winning solution to Hateful Memes Challenge , 2020, ArXiv.

[5]  Md. Zakir Hossain,et al.  A Comprehensive Survey of Deep Learning for Image Captioning , 2018, ACM Comput. Surv..

[6]  Jiasen Lu,et al.  Hierarchical Question-Image Co-Attention for Visual Question Answering , 2016, NIPS.

[7]  Khalil Sima'an,et al.  A Shared Task on Multimodal Machine Translation and Crosslingual Image Description , 2016, WMT.

[8]  Walter Daelemans,et al.  Automatic detection of cyberbullying in social media text , 2018, PloS one.

[9]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[10]  Jürgen Schmidhuber,et al.  Framewise phoneme classification with bidirectional LSTM and other neural network architectures , 2005, Neural Networks.

[11]  Ingmar Weber,et al.  Automated Hate Speech Detection and the Problem of Offensive Language , 2017, ICWSM.

[12]  Kathleen M. Carley,et al.  Tree LSTMs with Convolution Units to Predict Stance and Rumor Veracity in Social Media Conversations , 2019, ACL.

[13]  Matthijs J. Warrens,et al.  Inequalities between multi-rater kappas , 2010, Adv. Data Anal. Classif..

[14]  Eric Gilbert,et al.  Online Harassment and Content Moderation: The Case of Blocklists , 2018, ACM Trans. Comput. Hum. Interact..

[15]  Pushpak Bhattacharyya,et al.  All-in-One: A Deep Attentive Multi-task Learning Framework for Humour, Sarcasm, Offensive, Motivation, and Sentiment on Memes , 2020, AACL.

[16]  Jeffrey Pennington,et al.  GloVe: Global Vectors for Word Representation , 2014, EMNLP.

[17]  Huan Liu,et al.  Tracing Fake-News Footprints: Characterizing Social Media Messages by How They Propagate , 2018, WSDM.

[18]  Xiaogang Wang,et al.  Improving Deep Visual Representation for Person Re-identification by Global and Local Image-language Association , 2018, ECCV.

[19]  Ming-Hsuan Yang,et al.  Weakly Supervised Coupled Networks for Visual Sentiment Analysis , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[20]  Yun Fu,et al.  Visual Semantic Reasoning for Image-Text Matching , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[21]  Jinyoung Han,et al.  Rumor Propagation is Amplified by Echo Chambers in Social Media , 2020, Scientific Reports.

[22]  G. G. Stokes "J." , 1890, The New Yale Book of Quotations.

[23]  Mihael Arcan,et al.  Multimodal Meme Dataset (MultiOFF) for Identifying Offensive Content in Image and Text , 2020, TRAC.

[24]  Michael Wiegand,et al.  A Survey on Hate Speech Detection using Natural Language Processing , 2017, SocialNLP@EACL.

[25]  D. Gabard,et al.  Race , 1998, Encyclopedia of the UN Sustainable Development Goals.

[26]  Carolyn A. Lin,et al.  Defining Cyberbullying , 2017, Pediatrics.

[27]  Sérgio Nunes,et al.  A Survey on Automatic Detection of Hate Speech in Text , 2018, ACM Comput. Surv..

[28]  Qin Jin,et al.  Video Description Generation using Audio and Visual Cues , 2016, ICMR.

[29]  Chuang Gan,et al.  Neural-Symbolic VQA: Disentangling Reasoning from Vision and Language Understanding , 2018, NeurIPS.

[30]  Ziqi Zhang,et al.  Hate Speech Detection: A Solved Problem? The Challenging Case of Long Tail on Twitter , 2018, Semantic Web.

[31]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[32]  Andrew Jakubowicz,et al.  Alt_Right White Lite: Trolling, Hate Speech and Cyber Racism on Social Media , 2017 .

[33]  Cho-Jui Hsieh,et al.  VisualBERT: A Simple and Performant Baseline for Vision and Language , 2019, ArXiv.

[34]  John R. Hershey,et al.  Attention-Based Multimodal Fusion for Video Description , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[35]  Douwe Kiela,et al.  The Hateful Memes Challenge: Detecting Hate Speech in Multimodal Memes , 2020, NeurIPS.

[36]  Pietro Perona,et al.  Microsoft COCO: Common Objects in Context , 2014, ECCV.

[37]  Qi Wu,et al.  Visual question answering: A survey of methods and datasets , 2016, Comput. Vis. Image Underst..

[38]  Ke Li,et al.  FauxBuster: A Content-free Fauxtography Detector Using Social Media Comments , 2018, 2018 IEEE International Conference on Big Data (Big Data).

[39]  Daphney-Stavroula Zois,et al.  Cyberbullying Ends Here: Towards Robust Detection of Cyberbullying in Social Media , 2019, WWW.

[40]  Suhang Wang,et al.  Fake News Detection on Social Media: A Data Mining Perspective , 2017, SKDD.

[41]  Dirk Hovy,et al.  Hateful Symbols or Hateful People? Predictive Features for Hate Speech Detection on Twitter , 2016, NAACL.