Detecting the target of sarcasm is hard: Really??

Abstract Sarcasm target detection (identifying the target of mockery in a sarcastic sentence) is an emerging field in computational linguistics. Although there has been some research in this field, accurately identifying the target still remains problematic especially when the target of mockery is not presented in the text. In this paper, we propose a combination of a machine learning classifier and a deep learning model to extract the target of sarcasm from the text. First, we classify sarcastic sentences using machine learning, to determine whether a sarcastic sentence contains a target. Then we use a deep learning model from Aspect-Based Sentiment Analysis to extract the target. Our proposed system is evaluated on three publicly available data sets: sarcastic book snippets, sarcastic tweets, and sarcastic Reddit comments. Our evaluation results show that our approach achieves equal or better performance compared to the current state-of-the-art system, with an 18% improvement on the Reddit data set and similar scores on the Books and Tweets data sets. This is because our method is able to accurately identify when the target of sarcasm is not present. The primary challenge we identify, that is hindering the creation of a high accuracy classifier, is the lack of consistency among human annotators in identifying the target of sarcasm within standard ground-truth data sets.

[1]  Andrew Trotman,et al.  Detecting Target of Sarcasm using Ensemble Methods , 2019, ALTA.

[2]  Choochart Haruechaiyasak,et al.  Improving emotion classification in imbalanced YouTube dataset using SMOTE algorithm , 2015, 2015 2nd International Conference on Advanced Informatics: Concepts, Theory and Applications (ICAICTA).

[3]  Deniz Yuret,et al.  Transfer Learning for Low-Resource Neural Machine Translation , 2016, EMNLP.

[4]  Sanjeev Arora,et al.  A Simple but Tough-to-Beat Baseline for Sentence Embeddings , 2017, ICLR.

[5]  Dong Nguyen,et al.  Emo, love and god: making sense of Urban Dictionary, a crowd-sourced online dictionary , 2017, Royal Society Open Science.

[6]  Jia Song,et al.  A bi-directional sampling based on K-means method for imbalance text classification , 2016, 2016 IEEE/ACIS 15th International Conference on Computer and Information Science (ICIS).

[7]  Oksana Smal,et al.  POLITICAL DISCOURSE CONTENT ANALYSIS: A CRITICAL OVERVIEW OF A COMPUTERIZED TEXT ANALYSIS PROGRAM LINGUISTIC INQUIRY AND WORD COUNT (LIWC) , 2020, Naukovì zapiski Nacìonalʹnogo unìversitetu «Ostrozʹka akademìâ». Serìâ «Fìlologìâ».

[8]  Aijun An,et al.  Affective Representations for Sarcasm Detection , 2018, SIGIR.

[9]  Hoo-Chang Hoo-Chang Shin Shin,et al.  Deep Convolutional Neural Networks for Computer-Aided Detection: CNN Architectures, Dataset Characteristics and Transfer Learning , 2016, Ieee Transactions on Medical Imaging.

[10]  Luke S. Zettlemoyer,et al.  Deep Contextualized Word Representations , 2018, NAACL.

[11]  Lars Kotthoff,et al.  Automated Machine Learning: Methods, Systems, Challenges , 2019, The Springer Series on Challenges in Machine Learning.

[12]  Xiaocheng Feng,et al.  Effective LSTMs for Target-Dependent Sentiment Classification , 2015, COLING.

[13]  Yiming Yang,et al.  Transformer-XL: Attentive Language Models beyond a Fixed-Length Context , 2019, ACL.

[14]  Pushpak Bhattacharyya,et al.  How Challenging is Sarcasm versus Irony Classification?: A Study With a Dataset from English Literature , 2016, ALTA.

[15]  Rong Chen,et al.  Identify Severity Bug Report with Distribution Imbalance by CR-SMOTE and ELM , 2019, Int. J. Softw. Eng. Knowl. Eng..

[16]  Hai Jin,et al.  Graph Processing on GPUs , 2018, ACM Comput. Surv..

[17]  Anna Rumshisky,et al.  What’s in Your Embedding, And How It Predicts Task Performance , 2018, COLING.

[18]  M. Inés Torres,et al.  Extracting relevant knowledge for the detection of sarcasm and nastiness in the social web , 2014, Knowl. Based Syst..

[19]  Cindy K. Chung,et al.  Linguistic Inquiry and Word Count (LIWC): Pronounced “Luke,” . . . and Other Useful Facts , 2012 .

[20]  Dan Roth,et al.  Solving Hard Coreference Problems , 2019, NAACL.

[21]  Hideaki Hata,et al.  Sentiment Classification Using N-Gram Inverse Document Frequency and Automated Machine Learning , 2019, IEEE Software.

[22]  Walid Magdy,et al.  Exploring Author Context for Detecting Intended vs Perceived Sarcasm , 2019, ACL.

[23]  Eduardo Valle,et al.  Handling Inter-Annotator Agreement for Automated Skin Lesion Segmentation , 2019, ArXiv.

[24]  Julia Jorgensen,et al.  The functions of sarcastic irony in speech , 1996 .

[25]  Nina Wacholder,et al.  Identifying Sarcasm in Twitter: A Closer Look , 2011, ACL.

[26]  Matt Crane,et al.  Questionable Answers in Question Answering Research: Reproducibility and Variability of Published Results , 2018, TACL.

[27]  Johanna K. Kaakinen,et al.  The role of look-backs in the processing of written sarcasm , 2018, Memory & Cognition.

[28]  Ioannis Hatzilygeroudis,et al.  Recognizing emotions in text using ensemble of classifiers , 2016, Eng. Appl. Artif. Intell..

[29]  R. Gibbs The Poetics of Mind: Figurative Thought, Language, and Understanding , 1994 .

[30]  Yunfei Long,et al.  Inferring Affective Meanings of Words from Word Embedding , 2017, IEEE Transactions on Affective Computing.

[31]  Philipp Cimiano,et al.  An Impact Analysis of Features in a Classification Approach to Irony Detection in Product Reviews , 2014, WASSA@ACL.

[32]  Ming-Wei Chang,et al.  BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding , 2019, NAACL.

[33]  Nitesh V. Chawla,et al.  SMOTE: Synthetic Minority Over-sampling Technique , 2002, J. Artif. Intell. Res..

[34]  Hinrich Schütze,et al.  Learning Better Embeddings for Rare Words Using Distributional Representations , 2015, EMNLP.

[35]  David Reitter,et al.  Is Word Adoption a Grassroots Process? An Analysis of Reddit Communities , 2017, SBP-BRiMS.

[36]  Sanjay Kumar Jena,et al.  Parsing-based sarcasm sentiment recognition in Twitter data , 2015, 2015 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM).

[37]  Diego Molla,et al.  Overview of the 2019 ALTA Shared Task: Sarcasm Target Identification , 2019, ALTA.

[38]  Pushpak Bhattacharyya,et al.  Automatic Sarcasm Detection , 2016, ACM Comput. Surv..

[39]  Bingsheng He,et al.  Efficient Memory Management for GPU-based Deep Learning Systems , 2019, ArXiv.

[40]  Chun Chen,et al.  Opinion Word Expansion and Target Extraction through Double Propagation , 2011, CL.

[41]  Thomas Fang Zheng,et al.  Transfer learning for speech and language processing , 2015, 2015 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA).

[42]  Xiang Zhang,et al.  Character-level Convolutional Networks for Text Classification , 2015, NIPS.

[43]  Paolo Rosso,et al.  Irony detection via sentiment-based transfer learning , 2019, Inf. Process. Manag..

[44]  Diana Maynard,et al.  Who cares about Sarcastic Tweets? Investigating the Impact of Sarcasm on Sentiment Analysis. , 2014, LREC.

[45]  Tomas Mikolov,et al.  Bag of Tricks for Efficient Text Classification , 2016, EACL.

[46]  Pushpak Bhattacharyya,et al.  Automatic Identification of Sarcasm Target: An Introductory Approach , 2016, ArXiv.

[47]  Marc D. Pell,et al.  The sound of sarcasm , 2008, Speech Commun..

[48]  Kevin Gimpel,et al.  ALBERT: A Lite BERT for Self-supervised Learning of Language Representations , 2019, ICLR.

[49]  Li Zhao,et al.  Attention-based LSTM for Aspect-level Sentiment Classification , 2016, EMNLP.

[50]  Rada Mihalcea,et al.  CASCADE: Contextual Sarcasm Detection in Online Discussion Forums , 2018, COLING.

[51]  Liyana Shuib,et al.  Sarcasm identification in textual data: systematic review, research challenges and open directions , 2019, Artificial Intelligence Review.

[52]  P. Pexman It's Fascinating Research , 2008 .

[53]  Yoshua Bengio,et al.  Inference for the Generalization Error , 1999, Machine Learning.

[54]  Fan Min,et al.  Three-way decisions based feature fusion for Chinese irony detection , 2019, Int. J. Approx. Reason..

[55]  Debanjan Ghosh,et al.  "With 1 follower I must be AWESOME : P". Exploring the role of irony markers in irony recognition , 2018, ICWSM.

[56]  David Bamman,et al.  Contextualized Sarcasm Detection on Twitter , 2015, ICWSM.

[57]  Dan Sperber,et al.  Verbal irony: Pretense or echoic mention? , 1984 .

[58]  Jeffrey Pennington,et al.  GloVe: Global Vectors for Word Representation , 2014, EMNLP.

[59]  Atsuto Maki,et al.  A systematic study of the class imbalance problem in convolutional neural networks , 2017, Neural Networks.

[60]  Roman Klinger,et al.  An Empirical, Quantitative Analysis of the Differences Between Sarcasm and Irony , 2016, ESWC.

[61]  Paul Piwek,et al.  Rethinking the Agreement in Human Evaluation Tasks , 2018, COLING.

[62]  Byron C. Wallace,et al.  Modelling Context with User Embeddings for Sarcasm Detection in Social Media , 2016, CoNLL.

[63]  Fernando Bação,et al.  Oversampling for Imbalanced Learning Based on K-Means and SMOTE , 2017, Inf. Sci..

[64]  Tatsuya Kai Robust Control of a 3D Space Robot with an Initial Angular Momentum based on the Nonlinear Model Predictive Control Method , 2018 .

[65]  Reza Zafarani,et al.  Sarcasm Detection on Twitter: A Behavioral Modeling Approach , 2015, WSDM.

[66]  Aaron Klein,et al.  Efficient and Robust Automated Machine Learning , 2015, NIPS.

[67]  Eric Gilbert,et al.  The Bag of Communities: Identifying Abusive Behavior Online with Preexisting Internet Data , 2017, CHI.

[68]  David Yarowsky,et al.  Classifying latent user attributes in twitter , 2010, SMUC '10.

[69]  Gary King,et al.  ReLogit: Rare Events Logistic Regression , 2003 .

[70]  Hongwei Ge,et al.  Recognition of Ironic Sentences in Twitter using Attention-Based LSTM , 2018 .

[71]  Francisco Herrera,et al.  SMOTE for Learning from Imbalanced Data: Progress and Challenges, Marking the 15-year Anniversary , 2018, J. Artif. Intell. Res..

[72]  Tao Li,et al.  Aspect Based Sentiment Analysis with Gated Convolutional Networks , 2018, ACL.

[73]  Liyuan Liu,et al.  A2Text-Net: A Novel Deep Neural Network for Sarcasm Detection , 2019, 2019 IEEE First International Conference on Cognitive Machine Intelligence (CogMI).

[74]  Srijan Bansal,et al.  A deep-learning framework to detect sarcasm targets , 2019, EMNLP/IJCNLP.

[75]  R. Giora On Our Mind: Salience, Context, and Figurative Language , 2003 .

[76]  R. Kreuz,et al.  How to be sarcastic: The echoic reminder theory of verbal irony. , 1989 .

[77]  Zeerak Waseem,et al.  Are You a Racist or Am I Seeing Things? Annotator Influence on Hate Speech Detection on Twitter , 2016, NLP+CSS@EMNLP.

[78]  Anjana Gosain,et al.  Handling class imbalance problem using oversampling techniques: A review , 2017, 2017 International Conference on Advances in Computing, Communications and Informatics (ICACCI).

[79]  Pnina Fichman,et al.  Multidimensionality of online trolling behaviors , 2018, Inf. Soc..

[80]  Elisabetta Fersini,et al.  Detecting irony and sarcasm in microblogs: The role of expressive signals and ensemble classifiers , 2015, 2015 IEEE International Conference on Data Science and Advanced Analytics (DSAA).

[81]  Xing Fang,et al.  Toward multi-label sentiment analysis: a transfer learning based approach , 2020, Journal of Big Data.

[82]  Tomas Mikolov,et al.  Enriching Word Vectors with Subword Information , 2016, TACL.

[83]  Gary King,et al.  Logistic Regression in Rare Events Data , 2001, Political Analysis.

[84]  Tony Veale,et al.  Fracking Sarcasm using Neural Network , 2016, WASSA@NAACL-HLT.

[85]  Lidong Bing,et al.  Recurrent Attention Network on Memory for Aspect Sentiment Analysis , 2017, EMNLP.

[86]  P. Rockwell,et al.  Lower, Slower, Louder: Vocal Cues of Sarcasm , 2000 .

[87]  Qiang Yang,et al.  A Survey on Transfer Learning , 2010, IEEE Transactions on Knowledge and Data Engineering.

[88]  Mikhail Khodak,et al.  A Large Self-Annotated Corpus for Sarcasm , 2017, LREC.

[89]  Nan Hua,et al.  Universal Sentence Encoder for English , 2018, EMNLP.

[90]  M. Sabbagh Communicative Intentions and Language: Evidence from Right-Hemisphere Damage and Autism , 1999, Brain and Language.