Asking the Right Question: Inferring Advice-Seeking Intentions from Personal Narratives

People often share personal narratives in order to seek advice from others. To properly infer the narrator's intention, one needs to apply a certain degree of common sense and social intuition. To test the capabilities of NLP systems to recover such intuition, we introduce the new task of inferring what is the advice-seeking goal behind a personal narrative. We formulate this as a cloze test, where the goal is to identify which of two advice-seeking questions was removed from a given narrative. The main challenge in constructing this task is finding pairs of semantically plausible advice-seeking questions for given narratives. To address this challenge, we devise a method that exploits commonalities in experiences people share online to automatically extract pairs of questions that are appropriate candidates for the cloze task. This results in a dataset of over 20,000 personal narratives, each matched with a pair of related advice-seeking questions: one actually intended by the narrator, and the other one not. The dataset covers a very broad array of human experiences, from dating, to career options, to stolen iPads. We use human annotation to determine the degree to which the task relies on common sense and social intuition in addition to a semantic understanding of the narrative. By introducing several baselines for this new task we demonstrate its feasibility and identify avenues for better modeling the intention of the narrator.

[1]  Yejin Choi,et al.  Event2Mind: Commonsense Inference on Events, Intents, and Reactions , 2018, ACL.

[2]  Holger Schwenk,et al.  Supervised Learning of Universal Sentence Representations from Natural Language Inference Data , 2017, EMNLP.

[3]  Idan Szpektor,et al.  Learning from the past: answering new questions with past answers , 2012, WWW.

[4]  Yejin Choi,et al.  Modeling Naive Psychology of Characters in Simple Commonsense Stories , 2018, ACL.

[5]  Dan Goldwasser,et al.  Ask, and Shall You Receive? Understanding Desire Fulfillment in Natural Language Text , 2016, AAAI.

[6]  Danqi Chen,et al.  CoQA: A Conversational Question Answering Challenge , 2018, TACL.

[7]  Nathanael Chambers,et al.  A Corpus and Cloze Evaluation for Deeper Understanding of Commonsense Stories , 2016, NAACL.

[8]  Quan Z. Sheng,et al.  Detecting Duplicate Posts in Programming QA Communities via Latent Semantics and Association Rules , 2017, WWW.

[9]  Filip Radlinski,et al.  Inferring query intent from reformulations and clicks , 2010, WWW '10.

[10]  Jeffrey Pennington,et al.  GloVe: Global Vectors for Word Representation , 2014, EMNLP.

[11]  Lillian Lee,et al.  All Who Wander: On the Prevalence and Characteristics of Multi-community Engagement , 2015, WWW.

[12]  Alec Radford,et al.  Improving Language Understanding by Generative Pre-Training , 2018 .

[13]  Sanda M. Harabagiu,et al.  Questions And Intentions , 2008 .

[14]  Kathleen R. McKeown,et al.  Modeling Narrative Discourse , 2012 .

[15]  Kathy McKeown,et al.  Towards Automatic Detection of Narrative Structure , 2014, LREC.

[16]  Chris Dyer,et al.  The NarrativeQA Reading Comprehension Challenge , 2017, TACL.

[17]  Mark T. Maybury,et al.  Automatic Summarization , 2002, Computational Linguistics.

[18]  Filip Radlinski,et al.  A Theoretical Framework for Conversational Search , 2017, CHIIR.

[19]  J. Pennebaker.  Writing About Emotional Experiences as a Therapeutic Process , 1997 .

[20]  Eunsol Choi,et al.  QuAC: Question Answering in Context , 2018, EMNLP.

[21]  Hal Daumé,et al.  Learning to Ask Good Questions: Ranking Clarification Questions using Neural Expected Value of Perfect Information , 2018, ACL.

[22]  Elahe Rahimtoroghi,et al.  Modelling Protagonist Goals and Desires in First-Person Narrative , 2017, SIGDIAL Conference.

[23]  Charles Rossiter,et al.  Expressive writing: words that heal , 2016 .

[24]  Heinz-Martin Süß,et al.  The Cue Is the Key , 2015 .

[25]  Michael E. Bratman,et al.  Intention, Plans, and Practical Reason , 1991 .

[26]  Matthew Richardson,et al.  MCTest: A Challenge Dataset for the Open-Domain Machine Comprehension of Text , 2013, EMNLP.

[27]  Kathleen McKeown,et al.  Modeling Reportable Events as Turning Points in Narrative , 2015, EMNLP.

[28]  Guokun Lai,et al.  Large-scale Cloze Test Dataset Created by Teachers , 2017, EMNLP.

[29]  Reid Swanson,et al.  An Empirical Analysis of Subjectivity and Narrative Levels in Weblog Storytelling Across Cultures , 2017, Dialogue Discourse.

[30]  Meredith Ringel Morris,et al.  Collaborative search revisited , 2013, CSCW.

[31]  Minlie Huang,et al.  Story Ending Generation with Incremental Encoding and Commonsense Knowledge , 2018, AAAI.

[32]  Xinya Du,et al.  Learning to Ask: Neural Question Generation for Reading Comprehension , 2017, ACL.

[33]  Partha P. Talukdar,et al.  Discovering Response-Eliciting Factors in Social Question Answering : A Reddit Inspired Study , 2021, ICWSM.

[34]  Alexandra Meliou,et al.  SQuID: Semantic Similarity-Aware Query Intent Discovery , 2018, SIGMOD Conference.

[35]  Alessandro Moschitti,et al.  Supervised Clustering of Questions into Intents for Dialog System Applications , 2018, EMNLP.

[36]  J. Pennebaker,et al.  Forming a story: the health benefits of narrative. , 1999, Journal of clinical psychology.

[37]  Susan T. Dumais,et al.  Characterizing the value of personalizing search , 2007, SIGIR.

[38]  James W. Pennebaker,et al.  Expressive Writing: Words That Heal , 2014 .

[39]  Eric Horvitz,et al.  SearchTogether: an interface for collaborative web search , 2007, UIST.

[40]  Heinz-Martin Süß,et al.  New Findings About Social Intelligence , 2013 .

[41]  Nathanael Chambers,et al.  Unsupervised Learning of Narrative Event Chains , 2008, ACL.

[42]  Krys J. Kochut,et al.  Text Summarization Techniques: A Brief Survey , 2017, International Journal of Advanced Computer Science and Applications.

[43]  Hannah Rohde,et al.  Evaluating an Expectation-Driven Question-Under-Discussion Model of Discourse Interpretation , 2017 .