AI Cannot Understand Memes: Experiments with OCR and Facial Emotions

The increasing capabilities of Artificial Intelligence (AI), has led researchers and visionaries to think in the direction of machines outperforming humans by gaining intelligence equal to or greater than humans, which may not always have a positive impact on the society. AI gone rogue, and Technological Singularity are major concerns in academia as well as the industry. It is necessary to identify the limitations of machines and analyze their incompetence, which could draw a line between human and machine intelligence. Internet memes are an amalgam of pictures, videos, underlying messages, ideas, sentiments, humor, and experiences, hence the way an internet meme is perceived by a human may not be entirely how a machine comprehends it. In this paper, we present experimental evidence on how comprehending Internet Memes is a challenge for AI. We use a combination of Optical Character Recognition techniques like Tesseract, Pixel Link, and East Detector to extract text from the memes, and machine learning algorithms like Convolutional Neural Networks (CNN), Region-based Convolutional Neural Networks (RCNN), and Transfer Learning with pre-trained denseNet for assessing the textual and facial emotions combined.We evaluate the performance using Sensitivity and Specificity. Our results show that comprehending memes is indeed a challenging task, and hence a major limitation of AI. This research would be of utmost interest to researchers working in the areas of Artificial General Intelligence and Technological Singularity.

[1]  Bin Li,et al.  Facial expression recognition via ResNet-50 , 2021, International Journal of Cognitive Computing in Engineering.

[2]  Chase Cotton,et al.  A novel LSTM–CNN–grid search-based deep neural network for sentiment analysis , 2021, The Journal of Supercomputing.

[3]  Tu N. Nguyen,et al.  Deep learning-based facial emotion recognition for human–computer interaction applications , 2021 .

[4]  Yahia Said,et al.  Human emotion recognition based on facial expressions via deep learning on high-resolution images , 2021, Multimedia Tools and Applications.

[5]  Le Hoang Son,et al.  A new enhanced cyber security framework for medical cyber physical systems , 2021, SICS Software-Intensive Cyber-Physical Systems.

[6]  P. Gupta,et al.  Decoding Emotions in Text Using GloVe Embeddings , 2021, 2021 International Conference on Computing, Communication, and Intelligent Systems (ICCCIS).

[7]  Vikram Puri,et al.  A convolutional neural network (CNN) based ensemble model for exoplanet detection , 2021, Earth Science Informatics.

[8]  Xitao Ma,et al.  PIEED: Position information enhanced encoder-decoder framework for scene text recognition , 2021, Applied Intelligence.

[9]  Pushpak Bhattacharyya,et al.  A Multitask Framework to Detect Depression, Sentiment and Multi-label Emotion from Suicide Notes , 2021, Cognitive Computation.

[10]  Vikram Puri,et al.  Smart contract based policies for the Internet of Things , 2021, Clust. Comput..

[11]  Ajith Abraham,et al.  Bi-heuristic ant colony optimization-based approaches for traveling salesman problem , 2020, Soft Computing.

[12]  Dac-Nhuong Le,et al.  Global Forecasting Confirmed and Fatal Cases of COVID-19 Outbreak Using Autoregressive Integrated Moving Average Model , 2020, Frontiers in Public Health.

[13]  Subodh Wairya,et al.  Hybrid deep neural network with adaptive galactic swarm optimization for text extraction from scene images , 2020, Soft Comput..

[14]  Ishaani Priyadarshini,et al.  Intelligence in cyberspace: the road to cyber singularity , 2020, J. Exp. Theor. Artif. Intell..

[15]  Raghvendra Kumar,et al.  Performance evaluation of Botnet DDoS attack detection using machine learning , 2020, Evol. Intell..

[16]  Brojo Kishore Mishra,et al.  A Hybrid Action-Related K-Nearest Neighbour (HAR-KNN) Approach for Recommendation Systems , 2020, IEEE Access.

[17]  Ninad Mehendale,et al.  Facial emotion recognition using convolutional neural networks (FERC) , 2020, SN Applied Sciences.

[18]  Chase Cotton,et al.  Some Cyberpsychology Techniques to Distinguish Humans and Bots for Authentication , 2019, Advances in Intelligent Systems and Computing.

[19]  Chase Cotton,et al.  Internet Memes: A Novel Approach to Distinguish Humans and Bots for Authentication , 2019, Advances in Intelligent Systems and Computing.

[20]  Reda Alhajj,et al.  Emotion and sentiment analysis from Twitter text , 2019, J. Comput. Sci..

[21]  Tahir Mahmood,et al.  Multi-Attribute Multi-Perception Decision-Making Based on Generalized T-Spherical Fuzzy Weighted Aggregation Operators on Neutrosophic Sets , 2019, Mathematics.

[22]  Yu Liu,et al.  Extracting human emotions at different places based on facial expressions and spatial clustering analysis , 2019, Trans. GIS.

[23]  Mohamed Abdel-Basset,et al.  Deep Learning Approach for Software Maintainability Metrics Prediction , 2019, IEEE Access.

[24]  Puneet Agrawal,et al.  Understanding Emotions in Text Using Deep Learning and Big Data , 2019, Comput. Hum. Behav..

[25]  P. Geetha,et al.  Facial emotion detection using modified eyemap–mouthmap algorithm on an enhanced image and classification with tensorflow , 2019, The Visual Computer.

[26]  Martin Upchurch,et al.  Robots and AI at Work: The Prospects for Singularity , 2018, New Technology, Work and Employment.

[27]  Prathamesh Apte,et al.  Text-Based Emotion Analysis: Feature Selection Techniques and Approaches , 2018, Advances in Intelligent Systems and Computing.

[28]  Xuelong Li,et al.  PixelLink: Detecting Scene Text via Instance Segmentation , 2018, AAAI.

[29]  Alan S. Cowen,et al.  Self-report captures 27 distinct categories of emotion bridged by continuous gradients , 2017, Proceedings of the National Academy of Sciences.

[30]  Georgios Kambourakis,et al.  DDoS in the IoT: Mirai and Other Botnets , 2017, Computer.

[31]  Roman V. Yampolskiy,et al.  The Singularity May Be Near , 2017, Inf..

[32]  Shuchang Zhou,et al.  EAST: An Efficient and Accurate Scene Text Detector , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[33]  Barry A. T. Brown,et al.  The Social Life of Autonomous Cars , 2017, Computer.

[34]  Kilian Q. Weinberger,et al.  Densely Connected Convolutional Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[35]  K. Crawford.  Artificial Intelligence's White Guy Problem , 2016 .

[36]  Trevor Darrell,et al.  Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation , 2013, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[37]  Latanya Sweeney,et al.  Discrimination in online ad delivery , 2013, CACM.

[38]  R. Smith,et al.  An Overview of the Tesseract OCR Engine , 2007, Ninth International Conference on Document Analysis and Recognition (ICDAR 2007).

[39]  Nicu Sebe,et al.  Authentic facial expression analysis , 2004, Sixth IEEE International Conference on Automatic Face and Gesture Recognition, 2004. Proceedings..

[40]  L. Converso,et al.  Optical Character Recognition , 1990, TC.

[41]  Chase Cotton,et al.  Analyzing Some Elements of Technological Singularity Using Regression Methods , 2021 .

[42]  Mubashar Nazar Awan,et al.  TOP-Rank: A TopicalPostionRank for Extraction and Classification of Keyphrases in Text , 2021, Comput. Speech Lang..

[43]  B. Premjith,et al.  Emotion Detection in Hinglish(Hindi+English) Code-Mixed Social Media Text , 2020 .

[44]  Binh Thai Pham,et al.  Crime rate detection using social media of different crime locations and Twitter part-of-speech tagger with Brown clustering , 2020, J. Intell. Fuzzy Syst..

[45]  Mohamed Elhoseny,et al.  A Hybrid Artificial Intelligence and Internet of Things Model for Generation of Renewable Resource of Energy , 2019, IEEE Access.

[46]  Ishaani Priyadarshini,et al.  Cyber Security Risks in Robotics , 2017 .

[47]  Gavriel Salomon,et al.  T RANSFER OF LEARNING , 1992 .