Harnessing Artificial Intelligence to Improve the Quality of Answers in Online Question-answering Health Forums

ABSTRACT Quality of answers in health-related community-based question answering (HCQA) forums has been a concern for both users and forum administrators. We conducted a two-phase study to better understand the quality of answers in HCQA forums. First, we employed machine learning to examine the quality of health content. We validated our algorithmic quality ratings by comparing them with those of two physicians. Second, using data from Yahoo! Answers Health section, we examined the effect of the quality of the first answer on the quality of the subsequent answers. Our results suggest that the quality of the subsequent answers is impacted by the quality of the first displayed answer. We further show that the impact of the first displayed answer is larger when the answerers are more familiar with the forum but smaller when the forum provides tips for answering questions. Our study helps HCQA forums to improve the overall quality of answers by 1- creating an algorithmic solution that reliably measures the quality of answers, and 2- adjusting the order of existing answers to encourage higher quality subsequent answers. Our findings also extend the applicability of the order effect to online forums and provide evidence that experienced users would be more influenced by the order effect in such forums.

[1]  Jonas Mueller,et al.  Siamese Recurrent Architectures for Learning Sentence Similarity , 2016, AAAI.

[2]  Randall K. Minas,et al.  Sparking Creativity: Improving Electronic Brainstorming with Individual Cognitive Priming , 2013 .

[3]  D. Lasorsa.  Question-Order Effects in Surveys: The Case of Political Interest, News Attention, and Knowledge , 2003 .

[4]  P. Davies,et al.  Googling children's health: reliability of medical advice on the internet , 2010, Archives of Disease in Childhood.

[5]  Shawn P. Curley,et al.  Do Recommender Systems Manipulate Consumer Preferences? A Study of Anchoring Effects , 2013, Inf. Syst. Res..

[6]  V. Shute,et al.  Individual Difference in Repetition Priming and Its Relationship to Declarative Knowledge Acquisition. , 1993 .

[7]  Scott T. Grafton,et al.  Reductions in neural activity underlie behavioral components of repetition priming , 2005, Nature Neuroscience.

[8]  Leslie E. Papke,et al.  Panel data methods for fractional response variables with an application to test pass rates , 2008 .

[9]  Jeffrey Pomerantz,et al.  Evaluating and predicting answer quality in community QA , 2010, SIGIR.

[10]  Kurt A. Carlson,et al.  Improving Preference Assessment: Limiting the Effect of Context Through Pre-exposure to Attribute Levels , 2006, Manag. Sci..

[11]  Yong Tan,et al.  Feeling Blue? Go Online: An Empirical Study of Social Support among Patients , 2010, Inf. Syst. Res..

[12]  Daqing He,et al.  "I know what you are going through": Answers to informational questions about eating disorders in Yahoo! answers: A qualitative study , 2013, ASIST.

[13]  Langtao Chen,et al.  Fostering Participant Health Knowledge and Attitudes: An Econometric Study of a Chronic Disease-Focused Online Health Community , 2019, J. Manag. Inf. Syst..

[14]  Roger Tourangeau,et al.  Eye-Tracking Data: New Insights on Response Order Effects and Other Cognitive Shortcuts in Survey Responding. , 2008, Public opinion quarterly.

[15]  Kim Kirsner,et al.  The specificity of skill acquisition and transfer , 1997 .

[16]  Lei Zhang,et al.  Sentiment Analysis and Opinion Mining , 2017, Encyclopedia of Machine Learning and Data Mining.

[17]  Panagiotis G. Ipeirotis,et al.  Estimating the Helpfulness and Economic Impact of Product Reviews: Mining Text and Reviewer Characteristics , 2010, IEEE Transactions on Knowledge and Data Engineering.

[18]  Adam Worrall,et al.  Quality of health answers in social Q&A , 2012, ASIST.

[19]  David Schuff,et al.  What Makes a Helpful Review? A Study of Customer Reviews on Amazon.com , 2010 .

[20]  Beibei Li,et al.  Designing Ranking Systems for Hotels on Travel Search Engines by Mining User-Generated and Crowd-Sourced Content , 2011, Mark. Sci..

[21]  Douglas R. Vogel,et al.  How Doctors Gain Social and Economic Returns in Online Health-Care Communities: A Professional Capital Perspective , 2017, J. Manag. Inf. Syst..

[22]  Gilad Mishne,et al.  Finding high-quality content in social media , 2008, WSDM '08.

[23]  Liangfei Qiu,et al.  When a Doctor Knows, It Shows: An Empirical Analysis of Doctors' Responses in a Q&A Forum of an Online Healthcare Portal , 2019, Inf. Syst. Res..

[24]  Roger Tourangeau,et al.  Response Order Effects in Dichotomous Categorical Questions Presented Orally The Impact of Question and Respondent Attributes , 2007 .

[25]  Feng Xu,et al.  Detecting high-quality posts in community question answering sites , 2015, Inf. Sci..

[26]  Alexandra Sims,et al.  Safe infant sleep recommendations on the Internet: let's Google it. , 2012, The Journal of pediatrics.

[27]  Param Vir Singh,et al.  How to Attract and Retain Readers in Enterprise Blogging? , 2014, Inf. Syst. Res..

[28]  Don Fallis,et al.  Research Paper: Indicators of Accuracy of Consumer Health Information on the Internet: A Study of Indicators Relating to Information for Managing Fever in Children in the Home , 2002, J. Am. Medical Informatics Assoc..

[29]  Jan Marco Leimeister,et al.  Rate or Trade? Identifying Winning Ideas in Open Idea Sourcing , 2016, Inf. Syst. Res..

[30]  Ming Liu,et al.  Multimodal DBN for Predicting High-Quality Answers in cQA portals , 2013, ACL.

[31]  Jahna Otterbacher,et al.  'Helpfulness' in online communities: a measure of message quality , 2009, CHI.

[32]  Howard Frumkin,et al.  Medical information on the internet , 1997, Journal of General Internal Medicine.

[33]  A. Tversky,et al.  The framing of decisions and the psychology of choice. , 1981, Science.

[34]  Wenji Mao,et al.  Social Computing: From Social Informatics to Social Intelligence , 2007, IEEE Intell. Syst..

[35]  Manish Singh,et al.  Efficient Twitter sentiment classification using subjective distant supervision , 2017, 2017 9th International Conference on Communication Systems and Networks (COMSNETS).

[36]  Stephen Shaoyi Liao,et al.  Leveraging Financial Social Media Data for Corporate Fraud Detection , 2018, J. Manag. Inf. Syst..

[37]  Kai H. Lim,et al.  Brand Positioning Strategy Using Search Engine Marketing , 2010, MIS Q..

[38]  Ching-man Au Yeung,et al.  "Popularity Effect" in User-Generated Content: Evidence from Online Product Reviews , 2014, Inf. Syst. Res..

[39]  Elad Yom-Tov,et al.  The Quality of Online Answers to Parents Who Suspect That Their Child Has an Autism Spectrum Disorder , 2016, ICWSM.

[40]  David A. Schweidel,et al.  Synergy or Interference: The Effect of Product Placement on Commercial Break Audience Decline , 2014, Mark. Sci..

[41]  Lisa Gandy,et al.  CashTagNN: Using sentiment of tweets with CashTags to predict stock market prices , 2016, 2016 11th International Conference on Intelligent Systems: Theories and Applications (SITA).

[42]  H. Redmond,et al.  Breast cancer information on the internet: analysis of accessibility and accuracy. , 2012, Breast.

[43]  Bärbel Knäuper,et al.  The Impact of Age and Education on Response Order Effects in Attitude Measurement , 1999 .

[44]  Anindya Ghose,et al.  The Hidden Cost of Accommodating Crowdfunder Privacy Preferences: A Randomized Field Experiment , 2014, Manag. Sci..

[45]  Ioana Popescu,et al.  Advance Selling When Consumers Regret , 2012, Manag. Sci..

[46]  Norbert Schwarz,et al.  A Cognitive Model of Response-Order Effects in Survey Measurement , 1992 .

[47]  Qing Yang,et al.  Predicting Best Answerers for New Questions in Community Question Answering , 2010, WAIM.

[48]  Weiguo Fan,et al.  Measuring Customer Agility from Online Reviews Using Big Data Text Analytics , 2018, J. Manag. Inf. Syst..

[49]  W. Bruce Croft,et al.  A framework to predict the quality of answers with non-textual features , 2006, SIGIR.

[50]  Jon A. Krosnick,et al.  The Impact of Cognitive Sophistication and Attitude Importance on Response-Order and Question-Order Effects , 1992 .

[51]  R. Poldrack,et al.  Characterizing the neural mechanisms of skill learning and repetition priming: evidence from mirror reading. , 2001, Brain : a journal of neurology.

[52]  J. Krosnick,et al.  AN EVALUATION OF A COGNITIVE THEORY OF RESPONSE-ORDER EFFECTS IN SURVEY MEASUREMENT , 1987 .