Predicting Readers' Sarcasm Understandability by Modeling Gaze Behavior

Sarcasm understandability or the ability to understand textual sarcasm depends upon readers' language proficiency, social knowledge, mental state and attentiveness. We introduce a novel method to predict the sarcasm understandability of a reader. Presence of incongruity in textual sarcasm often elicits distinctive eye-movement behavior by human readers. By recording and analyzing the eye-gaze data, we show that eye-movement patterns vary when sarcasm is understood vis-a`-vis when it is not. Motivated by our observations, we propose a system for sarcasm understandability prediction using supervised machine learning. Our system relies on readers' eye-movement parameters and a few textual features, thence, is able to predict sarcasm understandability with an F-score of 93%, which demonstrates its efficacy. The availability of inexpensive embedded-eye-trackers on mobile devices creates avenues for applying such research which benefits web-content creators, review writers and social media analysts alike.

[1]  Shravan Vasishth,et al.  What is the scanpath signature of syntactic reanalysis , 2011 .

[2]  Bo Pang,et al.  A Sentimental Education: Sentiment Analysis Using Subjectivity Summarization Based on Minimum Cuts , 2004, ACL.

[3]  A. Katz,et al.  Are There Necessary Conditions for Inducing a Sense of Sarcastic Irony? , 2012 .

[4]  R. Giora On irony and negation , 1995 .

[5]  Pushpak Bhattacharyya,et al.  Measuring Sentiment Annotation Complexity of Text , 2014, ACL.

[6]  Penny M. Pexman,et al.  Context Incongruity and Irony Processing , 2003 .

[7]  Mário J. Silva,et al.  Clues for detecting irony in user-generated contents: oh...!! it's "so easy" ;-) , 2009, TSA@CIKM.

[8]  Alan Kennedy,et al.  Book Review: Eye Tracking: A Comprehensive Guide to Methods and Measures , 2016, Quarterly journal of experimental psychology.

[9]  R. Gibbs Comprehension and memory for nonliteral utterances: The problem of sarcastic indirect requests ☆ , 1986 .

[10]  M. Kutas,et al.  Reading senseless sentences: brain potentials reflect semantic incongruity. , 1980, Science.

[11]  J Aharon-Peretz,et al.  The neuroanatomical basis of understanding sarcasm and its relationship to social cognition. , 2005, Neuropsychology.

[12]  Ellen Riloff,et al.  Sarcasm as Contrast between a Positive Sentiment and Negative Situation , 2013, EMNLP.

[13]  Raja Parasuraman,et al.  Neuroergonomics: The Brain at Work , 2006 .

[14]  G. Miller,et al.  Test of the Mention Theory of Irony , 1984 .

[15]  Diana Maynard,et al.  Who cares about Sarcastic Tweets? Investigating the Impact of Sarcasm on Sentiment Analysis. , 2014, LREC.

[16]  Ari Rappoport,et al.  Semi-Supervised Recognition of Sarcasm in Twitter and Amazon , 2010, CoNLL.

[17]  Pushpak Bhattacharyya,et al.  Harnessing Context Incongruity for Sarcasm Detection , 2015, ACL.

[18]  Nina Wacholder,et al.  Identifying Sarcasm in Twitter: A Closer Look , 2011, ACL.

[19]  Xin Xu,et al.  Logistic Regression and Boosting for Labeled Bags of Instances , 2004, PAKDD.

[20]  R. P. Fishburne,et al.  Derivation of New Readability Formulas (Automated Readability Index, Fog Count and Flesch Reading Ease Formula) for Navy Enlisted Personnel , 1975 .

[21]  K. Rayner Eye movements in reading and information processing: 20 years of research. , 1998, Psychological bulletin.

[22]  Pushpak Bhattacharyya,et al.  Automatically Predicting Sentence Translation Difficulty , 2013, ACL.

[23]  Richard J. Gerrig,et al.  On the pretense theory of irony. , 1984, Journal of experimental psychology. General.

[24]  P. Gordon,et al.  The interplay of discourse congruence and lexical association during sentence processing: Evidence from ERPs and eye tracking. , 2007, Journal of memory and language.

[25]  Horacio Saggion,et al.  Modelling Sarcasm in Twitter, a Novel Approach , 2014, WASSA@ACL.

[26]  Antal van den Bosch,et al.  The perfect solution for detecting sarcasm in tweets #not , 2013, WASSA@NAACL-HLT.

[27]  H. Leuthold,et al.  Testing theories of irony processing using eye-tracking and ERPs. , 2014, Journal of experimental psychology. Learning, memory, and cognition.

[28]  Ian H. Witten,et al.  The WEKA data mining software: an update , 2009, SKDD.

[29]  Pascual Martínez-Gómez,et al.  Diagnosing Causes of Reading Difficulty using Bayesian Networks , 2013, IJCNLP.