Assessing Quality of Consumer Reviews in Mobile Application Markets: A Principal Component Analysis Approach

This study presents a simple, theory-based method for calculating a metric which reflects the quality of online consumer reviews in mobile application markets. Derived from prior online consumer review studies based on psychology, information quality, and economics literature, a metric for measuring online consumer review quality is developed. The metric is a weighted sum of three variables (Squared Star Rating, Log-transformed Word Count, and Sum of Squared Negative and Positive Sentiment), and weights for calculating the metric are estimated by using Principal Component Analysis (PCA) technique. Preliminary assessment of the proposed method shows that metrics computed by using the proposed method are positively correlated with helpfulness ranks of mobile application reviews in Google Play. However, PCA results show that one of the variables (i.e., sentiment) used for developing the metric did not load consistently on the first factor component. From the findings of the preliminary evaluation on the metric, limitations and future research directions of the proposed method are discussed.

[1]  Bin Gu,et al.  Do online reviews matter? - An empirical investigation of panel data , 2008, Decis. Support Syst..

[2]  Qing Cao,et al.  Exploring determinants of voting for the "helpfulness" of online user reviews: A text mining approach , 2011, Decis. Support Syst..

[3]  Han Zhang,et al.  Anxious or Angry? Effects of Discrete Emotions on the Perceived Helpfulness of Online Reviews , 2014, MIS Q..

[4]  Yue Pan,et al.  Born Unequal: A Study of the Helpfulness of User-Generated Product Reviews , 2011 .

[5]  Ming Zhou,et al.  Low-Quality Product Review Detection in Opinion Summarization , 2007, EMNLP.

[6]  I. Jolliffe Principal Component Analysis , 2002 .

[7]  S. Chaiken Heuristic versus systematic information processing and the use of source versus message cues in persuasion. , 1980 .

[8]  Soo-Min Kim,et al.  Automatically Assessing Review Helpfulness , 2006, EMNLP.

[9]  A. Tversky,et al.  Judgment under Uncertainty: Heuristics and Biases , 1974, Science.

[10]  Jeff T. Larsen,et al.  Negative information weighs more heavily on the brain: the negativity bias in evaluative categorizations. , 1998, Journal of personality and social psychology.

[11]  Saonee Sarker,et al.  Trends in Mobile Computing within the IS Discipline : A Ten-Year Retrospective Trends in Mobile Computing within the IS Discipline : A Ten-Year Retrospective , 2018 .

[12]  Jan J. Gerbrands,et al.  On the relationships between SVD, KLT and PCA , 1981, Pattern Recognit..

[13]  P. Herr,et al.  Effects of Word-of-Mouth and Product-Attribute Information on Persuasion: An Accessibility-Diagnosticity Perspective , 1991 .

[14]  Chien Chin Chen,et al.  Quality evaluation of product reviews using an information quality framework , 2011, Decis. Support Syst..

[15]  E. Clemons,et al.  When Online Reviews Meet Hyperdifferentiation: A Study of the Craft Beer Industry , 2006 .

[16]  Guanling Chen,et al.  AppJoy: personalized mobile application discovery , 2011, MobiSys '11.

[17]  Wayne D. Hoyer,et al.  An Integrative Framework for Understanding Two-sided Persuasion , 1994 .

[18]  Lorin M. Hitt,et al.  Self Selection and Information Role of Online Product Reviews , 2007, Inf. Syst. Res..

[19]  Sangaralingam Kajanan,et al.  A Mobile App Search Engine , 2013, Mob. Networks Appl..

[20]  David Schuff,et al.  What Makes a Helpful Review? A Study of Customer Reviews on Amazon.com , 2010 .

[21]  Makoto Nakayama,et al.  Are Amazon.com Online Review Helpfulness Ratings Biased or Not? , 2011, WEB.

[22]  JoongHo Ahn,et al.  Helpfulness of Online Consumer Reviews: Readers' Objectives and Review Cues , 2012, Int. J. Electron. Commer..

[23]  Monic Sun,et al.  How Does the Variance of Product Ratings Matter? , 2010, Manag. Sci..

[24]  Jahna Otterbacher,et al.  'Helpfulness' in online communities: a measure of message quality , 2009, CHI.

[25]  Jason Q. Zhang,et al.  When does electronic word-of-mouth matter? A study of consumer product reviews☆ , 2010 .

[26]  Ann E. Schlosser Can including pros and cons increase the helpfulness and persuasiveness of online reviews? The interactive effects of ratings and arguments ☆ , 2011 .

[27]  Elena García Barriocanal,et al.  Evaluating content quality and helpfulness of online product reviews: The interplay of review helpfulness vs. review content , 2012, Electron. Commer. Res. Appl..

[28]  Stuart J. Barnes,et al.  Mobile Business Research Published in 2000-2004: Emergence, Current Status, and Future Opportunities , 2006, Commun. Assoc. Inf. Syst..

[29]  Sangjae Lee,et al.  Predicting the helpfulness of online reviews using multilayer perceptron neural networks , 2014, Expert Syst. Appl..

[30]  Franziska Marquart,et al.  Communication and persuasion : central and peripheral routes to attitude change , 1988 .

[31]  Andrew D. Gershoff,et al.  Consumer Acceptance of Online Agent Advice: Extremity and Positivity Effects , 2003 .

[32]  Peter C. Neijens,et al.  "Highly Recommended!" The Content Characteristics and Perceived Usefulness of Online Consumer Reviews , 2011, J. Comput. Mediat. Commun..

[33]  Angelika Dimoka,et al.  The Nature and Role of Feedback Text Comments in Online Marketplaces: Implications for Trust Building, Price Premiums, and Seller Differentiation , 2006, Inf. Syst. Res..

[34]  Yubo Chen,et al.  Online Consumer Review: Word-of-Mouth as a New Element of Marketing Communication Mix , 2004, Manag. Sci..

[35]  Annie Lang,et al.  The limited capacity model of mediated message processing , 2000 .

[36]  Uzay Kaymak,et al.  Exploiting emoticons in sentiment analysis , 2013, SAC '13.

[37]  H. Gossen The Laws of Human Relations and the Rules of Human Action Derived Therefrom , 1983 .

[38]  Nina Wacholder,et al.  Identifying Sarcasm in Twitter: A Closer Look , 2011, ACL.

[39]  Andrew Whinston,et al.  The Dynamics of Online Word-of-Mouth and Product Sales: An Empirical Investigation of the Movie Industry , 2008 .

[40]  Ari Rappoport,et al.  RevRank: A Fully Unsupervised Algorithm for Selecting the Most Helpful Book Reviews , 2009, ICWSM.

[41]  Ellen Riloff,et al.  Sarcasm as Contrast between a Positive Sentiment and Negative Situation , 2013, EMNLP.

[42]  Chrysanthos Dellarocas,et al.  Exploring the value of online product reviews in forecasting sales: The case of motion pictures , 2007 .

[43]  Panagiotis G. Ipeirotis,et al.  Estimating the Helpfulness and Economic Impact of Product Reviews: Mining Text and Reviewer Characteristics , 2010, IEEE Transactions on Knowledge and Data Engineering.