Sarcasm Detection Using Soft Attention-Based Bidirectional Long Short-Term Memory Model With Convolution Network

A large community of research has been developed in recent years to analyze social media and social networks, with the aim of understanding, discovering insights, and exploiting the available information. The focus has shifted from conventional polarity classification to contemporary application-oriented fine-grained aspects such as, emotions, sarcasm, stance, rumor, and hate speech detection in the user-generated content. Detecting a sarcastic tone in natural language hinders the performance of sentiment analysis tasks. The majority of the studies on automatic sarcasm detection emphasize on the use of lexical, syntactic, or pragmatic features that are often unequivocally expressed through figurative literary devices such as words, emoticons, and exclamation marks. In this paper, we propose a deep learning model called sAtt-BLSTM convNet that is based on the hybrid of soft attention-based bidirectional long short-term memory (sAtt-BLSTM) and convolution neural network (convNet) applying global vectors for word representation (GLoVe) for building semantic word embeddings. In addition to the feature maps generated by the sAtt-BLSTM, punctuation-based auxiliary features are also merged into the convNet. The robustness of the proposed model is investigated using balanced (tweets from benchmark SemEval 2015 Task 11) and unbalanced (approximately 40000 random tweets using the Sarcasm Detector tool with 15000 sarcastic and 25000 non-sarcastic messages) datasets. An experimental study using the training- and test-set accuracy metrics is performed to compare the proposed deep neural model with convNet, LSTM, and bidirectional LSTM with/without attention and it is observed that the novel sAtt-BLSTM convNet model outperforms others with a superior sarcasm-classification accuracy of 97.87% for the Twitter dataset and 93.71% for the random-tweet dataset.

[1]  Le Hoang Son.  Generalized picture distance measure and applications to picture fuzzy clustering , 2016, Appl. Soft Comput..

[2]  Steven Bird,et al.  NLTK: The Natural Language Toolkit , 2002, ACL 2006.

[3]  Ashish Khanna,et al.  APD-JFAD: Accurate Prevention and Detection of Jelly Fish Attack in MANET , 2018, IEEE Access.

[4]  Le Hoang Son,et al.  THEORETICAL ANALYSIS OF PICTURE FUZZY CLUSTERING: CONVERGENCE AND PROPERTY , 2018, Journal of Computer Science and Cybernetics.

[5]  Yuanzhi Li,et al.  Convergence Analysis of Two-layer Neural Networks with ReLU Activation , 2017, NIPS.

[6]  Akshi Kumar,et al.  Sentiment Analysis: A Perspective on its Past, Present and Future , 2012 .

[7]  Salwani Abdullah,et al.  Approaches to Cross-Domain Sentiment Analysis: A Systematic Literature Review , 2017, IEEE Access.

[8]  Pushpak Bhattacharyya,et al.  How Do Cultural Differences Impact the Quality of Sarcasm Annotation?: A Case Study of Indian Annotators and American Text , 2016, LaTeCH@ACL.

[9]  Abdulmotaleb El Saddik,et al.  Sentiment Identification in Football-Specific Tweets , 2018, IEEE Access.

[10]  Yoon Kim,et al.  Convolutional Neural Networks for Sentence Classification , 2014, EMNLP.

[11]  Ali Farhadi,et al.  Bidirectional Attention Flow for Machine Comprehension , 2016, ICLR.

[12]  Isabelle Guyon,et al.  An Introduction to Feature Extraction , 2006, Feature Extraction.

[13]  MPS Bhatia,et al.  A PRIMER ON THE WEB INFORMATION RETRIEVAL PARADIGM , 2008 .

[14]  K. Saravanan,et al.  How to prevent maritime border collision for fisheries?-A design of Real-Time Automatic Identification System , 2018, Earth Science Informatics.

[15]  Francisco Chiclana,et al.  Dynamic structural neural network , 2018, J. Intell. Fuzzy Syst..

[16]  Yoshua Bengio,et al.  Show, Attend and Tell: Neural Image Caption Generation with Visual Attention , 2015, ICML.

[17]  S. N. Sivanandam,et al.  Principles of soft computing , 2011 .

[18]  Mike Thelwall,et al.  Topic-based sentiment analysis for the social web: The role of mood and issue-related words , 2013, J. Assoc. Inf. Sci. Technol..

[19]  Rajiv Kapoor,et al.  Boosting performance of power quality event identification with KL Divergence measure and standard deviation , 2018, Measurement.

[20]  Le Hoang Son,et al.  Novel fuzzy clustering scheme for 3D wireless sensor networks , 2017, Appl. Soft Comput..

[21]  Akshi Kumar,et al.  Sentiment Analysis on Twitter , 2012 .

[22]  Shahid Shayaa,et al.  Sentiment Analysis of Big Data: Methods, Applications, and Open Challenges , 2018, IEEE Access.

[23]  Le Hoang Son,et al.  A Novel Multiple Fuzzy Clustering Method Based on Internal Clustering Validation Measures with Gradient Descent , 2015, International Journal of Fuzzy Systems.

[24]  Le Hoang Son,et al.  Some novel hybrid forecast methods based on picture fuzzy clustering for weather nowcasting from satellite image sequences , 2016, Applied Intelligence.

[25]  Le Hoang Son,et al.  Improving lifetime and network connections of 3D wireless sensor networks based on fuzzy clustering and particle swarm optimization , 2018, Wirel. Networks.

[26]  Navdeep Jaitly,et al.  Hybrid speech recognition with Deep Bidirectional LSTM , 2013, 2013 IEEE Workshop on Automatic Speech Recognition and Understanding.

[27]  Mingmin Chi,et al.  Long Short-Term Memory With Quadratic Connections in Recursive Neural Networks for Representing Compositional Semantics , 2017, IEEE Access.

[28]  David M. W. Powers,et al.  Evaluation: from precision, recall and F-measure to ROC, informedness, markedness and correlation , 2011, ArXiv.

[29]  Francisco Chiclana,et al.  A new fusion of salp swarm with sine cosine for optimization of non-linear functions , 2019, Engineering with Computers.

[30]  Jianqiang Li,et al.  Lexicon-Enhanced LSTM With Attention for General Sentiment Analysis , 2018, IEEE Access.

[31]  Manju Khari,et al.  Collaborative handshaking approaches between internet of computing and internet of things towards a smart world: a review from 2009–2017 , 2018, Telecommun. Syst..

[32]  Rajiv Kapoor,et al.  New scheme for underwater acoustically wireless transmission using direct sequence code division multiple access in MIMO systems , 2019, Wirel. Networks.

[33]  Mumtaz Ali,et al.  A novel approach for fuzzy clustering based on neutrosophic association matrix , 2019, Computers & Industrial Engineering.

[34]  Byron C. Wallace,et al.  Modelling Context with User Embeddings for Sarcasm Detection in Social Media , 2016, CoNLL.

[35]  Ka-Chun Wong,et al.  Verbal aggression detection on Twitter comments: convolutional neural network for short-text sentiment analysis , 2018, Neural Computing and Applications.

[36]  Long Tian,et al.  Combining Convolution Neural Network and Bidirectional Gated Recurrent Unit for Sentence Semantic Classification , 2018, IEEE Access.

[37]  D. Jude Hemanth,et al.  Brain signal based human emotion analysis by circular back propagation and Deep Kohonen Neural Networks , 2018, Comput. Electr. Eng..

[38]  Jürgen Schmidhuber,et al.  Learning to forget: continual prediction with LSTM , 1999 .

[39]  Christopher D. Manning,et al.  Effective Approaches to Attention-based Neural Machine Translation , 2015, EMNLP.

[40]  Le Hoang Son,et al.  Real-time water quality monitoring using Internet of Things in SCADA , 2018, Environmental Monitoring and Assessment.

[41]  Sanjay Kumar Jena,et al.  Sarcastic sentiment detection in tweets streamed in real time: a big data approach , 2016, Digit. Commun. Networks.

[42]  Antoanela Naaji,et al.  A Modified Deep Convolutional Neural Network for Abnormal Brain Image Classification , 2019, IEEE Access.

[43]  Eric Gilbert,et al.  VADER: A Parsimonious Rule-Based Model for Sentiment Analysis of Social Media Text , 2014, ICWSM.

[44]  Tran Manh Tuan,et al.  A cooperative semi-supervised fuzzy clustering framework for dental X-ray image segmentation , 2016, Expert Syst. Appl..

[45]  Ellen Riloff,et al.  Sarcasm as Contrast between a Positive Sentiment and Negative Situation , 2013, EMNLP.

[46]  Elisabeth Camp.  Sarcasm, Pretense, and The Semantics/ Pragmatics Distinction ∗ , 2012 .

[47]  Hamido Fujita,et al.  Neural-fuzzy with representative sets for prediction of student performance , 2018, Applied Intelligence.

[48]  Ari Rappoport,et al.  Enhanced Sentiment Learning Using Twitter Hashtags and Smileys , 2010, COLING.

[49]  Tong Li,et al.  Sentiment Classification Based on Information Geometry and Deep Belief Networks , 2018, IEEE Access.

[50]  Erik Cambria,et al.  The CLSA Model: A Novel Framework for Concept-Level Sentiment Analysis , 2015, CICLing.

[51]  Steven Bird,et al.  NLTK: The Natural Language Toolkit , 2002, ACL.

[52]  Paolo Rosso,et al.  SemEval-2015 Task 11: Sentiment Analysis of Figurative Language in Twitter , 2015, *SEMEVAL.

[53]  Yue Zhang,et al.  Tweet Sarcasm Detection Using Deep Neural Network , 2016, COLING.

[54]  Gui Xiaolin,et al.  Deep Convolution Neural Networks for Twitter Sentiment Analysis , 2018, IEEE Access.

[55]  Mumtaz Ali,et al.  A Novel Clustering Algorithm in a Neutrosophic Recommender System for Medical Diagnosis , 2017, Cognitive Computation.

[56]  Martin F. Porter,et al.  An algorithm for suffix stripping , 1997, Program.

[57]  Le Hoang Son,et al.  A novel automatic picture fuzzy clustering method based on particle swarm optimization and picture composite cardinality , 2016, Knowl. Based Syst..

[58]  Rajiv Kapoor,et al.  Detection of Power Quality Event using Histogram of Oriented Gradients and Support Vector Machine , 2018 .

[59]  Tomoaki Ohtsuki,et al.  A Pattern-Based Approach for Sarcasm Detection on Twitter , 2016, IEEE Access.

[60]  Tran Manh Tuan,et al.  Dental segmentation from X-ray images using semi-supervised fuzzy clustering with spatial constraints , 2017, Eng. Appl. Artif. Intell..

[61]  Arunima Jaiswal,et al.  Empirical Study of Twitter and Tumblr for Sentiment Analysis using Soft Computing Techniques , 2022 .

[62]  Iyad Rahwan,et al.  Using millions of emoji occurrences to learn any-domain representations for detecting sentiment, emotion and sarcasm , 2017, EMNLP.

[63]  Jui-Jen Chou,et al.  Text Mining Analysis of Teaching Evaluation Questionnaires for the Selection of Outstanding Teaching Faculty Members , 2018, IEEE Access.

[64]  Akshi Kumar,et al.  Systematic literature review of sentiment analysis on Twitter using soft computing techniques , 2019, Concurr. Comput. Pract. Exp..

[65]  Shahrokh Valaee,et al.  Recent Advances in Recurrent Neural Networks , 2017, ArXiv.

[66]  Preslav Nakov,et al.  SemEval-2013 Task 2: Sentiment Analysis in Twitter , 2013, *SEMEVAL.

[67]  Le Hoang Son,et al.  Picture fuzzy clustering for complex data , 2016, Eng. Appl. Artif. Intell..

[68]  Le Hoang Son,et al.  Tune Up Fuzzy C-Means for Big Data: Some Novel Hybrid Clustering Algorithms Based on Initial Selection and Incremental Clustering , 2017, Int. J. Fuzzy Syst..

[69]  Jin Liu,et al.  Attention-Based Memory Network for Text Sentiment Classification , 2018, IEEE Access.

[70]  Tomoaki Ohtsuki,et al.  Multi-Class Sentiment Analysis in Twitter: What if Classification is Not the Answer , 2018, IEEE Access.

[71]  David M. Pennock,et al.  Mining the peanut gallery: opinion extraction and semantic classification of product reviews , 2003, WWW '03.

[72]  Le Hoang Son.  A novel kernel fuzzy clustering algorithm for Geo-Demographic Analysis , 2015, Inf. Sci..

[73]  J. Anitha,et al.  Diabetic Retinopathy Diagnosis from Retinal Images Using Modified Hopfield Neural Network , 2018, Journal of Medical Systems.

[74]  Le Hoang Son,et al.  Picture fuzzy clustering: a new computational intelligence method , 2016, Soft Comput..

[75]  Tony Veale,et al.  Fracking Sarcasm using Neural Network , 2016, WASSA@NAACL-HLT.

[76]  Ping-Feng Pai,et al.  Predicting Vehicle Sales by Sentiment Analysis of Twitter Data and Stock Market Values , 2018, IEEE Access.

[77]  Tomoaki Ohtsuki,et al.  A Pattern-Based Approach for Multi-Class Sentiment Analysis in Twitter , 2017, IEEE Access.

[78]  Jeffrey Pennington,et al.  GloVe: Global Vectors for Word Representation , 2014, EMNLP.

[79]  Karan Singh,et al.  Congestion control in wireless sensor networks by hybrid multi-objective optimization algorithm , 2018, Comput. Networks.

[80]  K. Saravanan,et al.  FD-AOMDV: fault-tolerant disjoint ad-hoc on-demand multipath distance vector routing algorithm in mobile ad-hoc networks , 2018, J. Ambient Intell. Humaniz. Comput..