Don’t Let Me Be Misunderstood:Comparing Intentions and Perceptions in Online Discussions

Discourse involves two perspectives: a person’s intention in making an utterance and others’ perception of that utterance. The misalignment between these perspectives can lead to undesirable outcomes, such as misunderstandings, low productivity and even overt strife. In this work, we present a computational framework for exploring and comparing both perspectives in online public discussions. We combine logged data about public comments on Facebook with a survey of over 16,000 people about their intentions in writing these comments or about their perceptions of comments that others had written. Unlike previous studies of online discussions that have largely relied on third-party labels to quantify properties such as sentiment and subjectivity, our approach also directly captures what the speakers actually intended when writing their comments. In particular, our analysis focuses on judgments of whether a comment is stating a fact or an opinion, since these concepts were shown to be often confused. We show that intentions and perceptions diverge in consequential ways. People are more likely to perceive opinions than to intend them, and linguistic cues that signal how an utterance is intended can differ from those that signal how it will be perceived. Further, this misalignment between intentions and perceptions can be linked to the future health of a conversation: when a comment whose author intended to share a fact is misperceived as sharing an opinion, the subsequent conversation is more likely to derail into uncivil behavior than when the comment is perceived as intended. Altogether, these findings may inform the design of discussion platforms that better promote positive interactions.

[1]  Richard Valliant,et al.  Poststratification and Conditional Variance Estimation , 1993 .

[2]  Bing Liu,et al.  Sentiment Analysis and Subjectivity , 2010, Handbook of Natural Language Processing.

[3]  Vitor R. Carvalho Modeling Intention in Email - Speech Acts, Information Leaks and Recommendation Models , 2011, Studies in Computational Intelligence.

[4]  David Bamman,et al.  Contextualized Sarcasm Detection on Twitter , 2015, ICWSM.

[5]  Martha Larson,et al.  Intent and its discontents: the user at the wheel of the online video search engine , 2012, ACM Multimedia.

[6]  Cristian Danescu-Niculescu-Mizil,et al.  Conversations Gone Awry: Detecting Early Signs of Conversational Failure , 2018, ACL.

[7]  Candace L. Sidner,et al.  Attention, Intentions, and the Structure of Discourse , 1986, CL.

[8]  Joshua A. Tucker,et al.  Less than you think: Prevalence and predictors of fake news dissemination on Facebook , 2019, Science Advances.

[9]  Raffaele Filieri What makes an online consumer review trustworthy , 2016 .

[10]  Jure Leskovec,et al.  Large-scale Analysis of Counseling Conversations: An Application of Natural Language Processing to Mental Health , 2016, TACL.

[11]  D. Tannen Conversational Style: Analyzing Talk Among Friends , 1984 .

[12]  Jon M. Kleinberg,et al.  Characterizing and curating conversation threads: expansion, focus, volume, re-entry , 2013, WSDM.

[13]  Cristian Danescu-Niculescu-Mizil,et al.  Asking too much? The rhetorical role of questions in political discourse , 2017, EMNLP.

[14]  Nina Wacholder,et al.  Identifying Sarcasm in Twitter: A Closer Look , 2011, ACL.

[15]  Yoon Kim,et al.  Convolutional Neural Networks for Sentence Classification , 2014, EMNLP.

[16]  Lei Gao,et al.  Detecting Online Hate Speech Using Context Aware Models , 2017, RANLP.

[17]  Hong Yu,et al.  Towards Answering Opinion Questions: Separating Facts from Opinions and Identifying the Polarity of Opinion Sentences , 2003, EMNLP.

[18]  Michael Gamon,et al.  Predicting Responses to Microblog Posts , 2012, NAACL.

[19]  Paul N. Bennett,et al.  Context-Aware Intent Identification in Email Conversations , 2019, SIGIR.

[20]  Janyce Wiebe,et al.  Learning Subjective Language , 2004, CL.

[21]  Yulan He,et al.  Sentence Subjectivity Detection with Weakly-Supervised Learning , 2011, IJCNLP.

[22]  Vitor R. Carvalho Email Information Leaks , 2011 .

[23]  Meredith Ringel Morris,et al.  What do people ask their social networks, and why?: a survey study of status message q&a behavior , 2010, CHI.

[24]  Ping Liu,et al.  Forecasting the presence and intensity of hostility on Instagram using linguistic and social features , 2018, ICWSM.

[25]  Lior Rokach,et al.  Identifying Informational vs. Conversational Questions on Community Question Answering Archives , 2018, WSDM.

[26]  Seunga Venus Jin,et al.  "A Match Made...Online?" The Effects of User-Generated Online Dater Profile Types (Free-Spirited Versus Uptight) on Other Users' Perception of Trustworthiness, Interpersonal Attraction, and Personality , 2015, Cyberpsychology Behav. Soc. Netw..

[27]  Daniel Gayo-Avello,et al.  Survey and evaluation of query intent detection methods , 2009, WSCD '09.

[28]  Claire Cardie,et al.  Annotating Expressions of Opinions and Emotions in Language , 2005, Lang. Resour. Evaluation.

[29]  Bernard J. Jansen,et al.  A Taxonomy for Classifying Questions Asked in Social Question and Answering , 2015, CHI Extended Abstracts.

[30]  Giuseppe Riccardi,et al.  Simultaneous dialog act segmentation and classification from human-human spoken conversations , 2011, 2011 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[31]  Jason Weston,et al.  Natural Language Processing (Almost) from Scratch , 2011, J. Mach. Learn. Res..

[32]  Santosh Regmi,et al.  What Make Facts Stand Out from Opinions? Distinguishing Facts from Opinions in News Media , 2015 .

[33]  H. McKee "YOUR VIEWS SHOWED TRUE IGNORANCE!!!": (Mis)Communication in an Online Interracial Discussion Forum , 2002 .

[34]  Janyce Wiebe,et al.  Development and Use of a Gold-Standard Data Set for Subjectivity Classifications , 1999, ACL.

[35]  Tan Hoi Piew,et al.  The Effects of Shopping Orientations, Online Trust and Prior Online Purchase Experience toward Customers’ Online Purchase Intention , 2010 .

[36]  A. Pentland,et al.  Thin slices of negotiation: predicting outcomes from conversational dynamics within the first 5 minutes. , 2007, The Journal of applied psychology.

[37]  S. Sundar,et al.  Effect of Source Attribution on Perception of Online News Stories , 1998 .

[38]  Cristian Danescu-Niculescu-Mizil,et al.  Trajectories of Blocked Community Members: Redemption, Recidivism and Departure , 2019, WWW.

[39]  Andrei Broder,et al.  A taxonomy of web search , 2002, SIGF.

[40]  Benno Stein,et al.  Measuring the quality of web content using factual information , 2012, WebQuality '12.

[41]  Penelope Brown,et al.  Politeness: Some Universals in Language Usage , 1989 .

[42]  F. Maxwell Harper,et al.  Facts or friends?: distinguishing informational and conversational questions in social Q&A sites , 2009, CHI.

[43]  Michael Granitzer,et al.  Objectivity classification in online media , 2010, HT '10.

[44]  Hannah Rohde,et al.  Evaluating an Expectation-Driven Question-Under-Discussion Model of Discourse Interpretation , 2017 .

[45]  Ravi Kumar,et al.  Dynamics of conversations , 2010, KDD.

[46]  Victor Corral-Verdugo The Effect of Examples and Gender on Third Graders' Ability to Distinguish Environmental Facts from Opinions , 1993 .

[47]  Charles L. A. Clarke,et al.  Classifying and Characterizing Query Intent , 2009, ECIR.

[48]  Amy Bruckman,et al.  "Did You Suspect the Post Would be Removed?" , 2019, Proc. ACM Hum. Comput. Interact..

[49]  L M Giambra,et al.  Curiosity and stimulation seeking across the adult life span: cross-sectional and 6- to 8-year longitudinal findings. , 1992, Psychology and aging.

[50]  Nigel K. L. Pope,et al.  Buying or browsing? An exploration of shopping orientations and online purchase intention , 2003 .

[51]  Mitchell Rabinowitz,et al.  Distinguishing facts from beliefs: fuzzy categories , 2013 .

[52]  Claire Cardie,et al.  Recognizing and Organizing Opinions Expressed in the World Press , 2003, New Directions in Question Answering.

[53]  Giuseppe Carenini,et al.  Subjectivity detection in spoken and written conversations , 2010, Natural Language Engineering.

[54]  Evita March,et al.  The dark side of Facebook®: The Dark Tetrad, negative social potency, and trolling behaviours , 2016 .

[55]  Ming-Wei Chang,et al.  BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding , 2019, NAACL.

[56]  John Uebersax,et al.  Statistical Modeling of Expert Ratings on Medical Treatment Appropriateness , 1993 .

[57]  Michael Gamon,et al.  Actionable Email Intent Modeling with Reparametrized RNNs , 2017, AAAI.

[58]  Ria Verleur,et al.  Flaming on YouTube , 2010, Comput. Hum. Behav..

[59]  Davide Buscaldi,et al.  From humor recognition to irony detection: The figurative language of social media , 2012, Data Knowl. Eng..

[60]  Ruohui Wang,et al.  Edge Detection Using Convolutional Neural Network , 2016, ISNN.

[61]  Debanjan Ghosh,et al.  The Role of Conversation Context for Sarcasm Detection in Online Interactions , 2017, SIGDIAL Conference.

[62]  David García,et al.  Generative models of online discussion threads: state of the art and research challenges , 2017, Journal of Internet Services and Applications.

[63]  Marcel Creemers,et al.  Predicting online purchase behavior: replications and tests of competing models , 2001, Proceedings of the 34th Annual Hawaii International Conference on System Sciences.

[64]  Jack Hessel,et al.  Something’s Brewing! Early Prediction of Controversy-causing Posts from Discussion Features , 2019, NAACL.