Mine Your Own Business: Market-Structure Surveillance Through Text Mining

Web 2.0 provides gathering places for Internet users in blogs, forums, and chat rooms. These gathering places leave footprints in the form of colossal amounts of data regarding consumers' thoughts, beliefs, experiences, and even interactions. In this paper, we propose an approach for firms to explore online user-generated content and “listen” to what customers write about their and their competitors' products. Our objective is to convert the user-generated content to market structures and competitive landscape insights. The difficulty in obtaining such market-structure insights from online user-generated content is that consumers' postings are often not easy to syndicate. To address these issues, we employ a text-mining approach and combine it with semantic network analysis tools. We demonstrate this approach using two cases---sedan cars and diabetes drugs---generating market-structure perceptual maps and meaningful insights without interviewing a single consumer. We compare a market structure based on user-generated content data with a market structure derived from more traditional sales and survey-based data to establish validity and highlight meaningful differences.

[1]  Ronen Feldman,et al.  A Systematic Cross-Comparison of Sequence Classifiers , 2006, SDM.

[2]  K. Freedland,et al.  The prevalence of comorbid depression in adults with diabetes: a meta-analysis. , 2001, Diabetes care.

[3]  Erik Duval,et al.  Quantitative analysis of user-generated content on the Web , 2008 .

[4]  Jen-Hung Huang,et al.  Herding in online product choice , 2006 .

[5]  H. Vroman.  The Loyalty Effect: The Hidden Force Behind Growth, Profits, and Lasting Value , 1996 .

[6]  Michael McGill,et al.  Introduction to Modern Information Retrieval , 1983 .

[7]  Bing Liu,et al.  Opinion Mining and Sentiment Analysis , 2011 .

[8]  Michael L. Littman,et al.  Measuring praise and criticism: Inference of semantic orientation from association , 2003, TOIS.

[9]  J. Eliashberg,et al.  MOVIEMOD: An Implementable Decision-Support System for Prerelease Market Evaluation of Motion Pictures , 2000 .

[10]  Gerard J. Tellis,et al.  Does Chatter Really Matter? Dynamics of User-Generated Content and Stock Performance , 2011, Mark. Sci..

[11]  Matthew Hurst,et al.  Deriving marketing intelligence from online discussion , 2005, KDD '05.

[12]  Lei Zhang,et al.  Entity discovery and assignment for opinion mining applications , 2009, KDD.

[13]  Bo Xu,et al.  Product Named Entity Recognition Based on Hierarchical Hidden Markov Model , 2005, SIGHAN@IJCNLP 2005.

[14]  David Godes,et al.  Using Online Conversations to Study Word-of-Mouth Communication , 2004 .

[15]  David M. Pennock,et al.  Mining the peanut gallery: opinion extraction and semantic classification of product reviews , 2003, WWW '03.

[16]  David G. Hendry,et al.  An informal information-seeking environment , 1997 .

[17]  Chrysanthos Dellarocas,et al.  Exploring the value of online product reviews in forecasting sales: The case of motion pictures , 2007 .

[18]  R. Feldman,et al.  Using text mining to analyze user forums , 2008, 2008 International Conference on Service Systems and Service Management.

[19]  Chrysanthos Dellarocas,et al.  Strategic Manipulation of Internet Opinion Forums: Implications for Consumers and Firms , 2004, Manag. Sci..

[20]  M. Callon,et al.  Mapping the Dynamics of Science and Technology , 1986 .

[21]  Panagiotis G. Ipeirotis,et al.  Estimating the Helpfulness and Economic Impact of Product Reviews: Mining Text and Reviewer Characteristics , 2010, IEEE Transactions on Knowledge and Data Engineering.

[22]  Satoru Kawai,et al.  An Algorithm for Drawing General Undirected Graphs , 1989, Inf. Process. Lett..

[23]  Eric T. Bradlow,et al.  Automated Marketing Research Using Online Customer Reviews , 2011 .

[24]  Yong Liu.  Word-of-Mouth for Movies: Its Dynamics and Impact on Box Office Revenue , 2006 .

[25]  David Krackhardt,et al.  PREDICTING WITH NETWORKS: NONPARAMETRIC MULTIPLE REGRESSION ANALYSIS OF DYADIC DATA * , 1988 .

[26]  John R. Hauser,et al.  Testing Competitive Market Structures , 1984 .

[27]  D. Swanson.  Migraine and Magnesium: Eleven Neglected Connections , 2015, Perspectives in biology and medicine.

[28]  Z. John Zhang,et al.  From Story Line to Box Office: A New Approach for Green-Lighting Movie Scripts , 2007, Manag. Sci..

[29]  I. Edwards,et al.  Adverse drug reactions: definitions, diagnosis, and management , 2000, The Lancet.

[30]  Giacomo Mauro DAriano.  Brand Equity and Advertising: Advertising's Role in Building Strong Brands. , 1994 .

[31]  Walt Detmar Meurers,et al.  Head-driven phrase structure grammar: linguistic approach, formal foundations, and computational realization , 2006 .

[32]  Yehuda Lindell,et al.  Text Mining at the Term Level , 1998, PKDD.

[33]  Vithala R. Rao,et al.  Inference of Hierarchical Choice Processes from Panel Data , 1981 .

[34]  Pei-Yu Sharon Chen,et al.  The Impact of Online Recommendations and Consumer Feedback on Sales , 2004, ICIS.

[35]  Deborah Roedder John,et al.  Brand Concept Maps: A Methodology for Identifying Brand Association Networks , 2006 .

[36]  Beibei Li,et al.  Designing Ranking Systems for Hotels on Travel Search Engines by Mining User-Generated and Crowd-Sourced Content , 2011, Mark. Sci..

[37]  Kristina Nilsson,et al.  Towards automatic recognition of product names: an exploratory study of brand names in economic texts , 2005, NODALIDA.

[38]  Ronen Feldman,et al.  Unsupervised Lexicon Acquisition for HPSG-Based Relation Extraction , 2011, IJCAI.

[39]  Irina Ionova,et al.  Chrysler and J. D. Power: Pioneering Scientific Price Customization in the Automobile Industry , 2008, Interfaces.

[40]  M. Callon,et al.  Mapping the dynamics of science and technology : sociology of science in the real world , 1988 .

[41]  Dawn Iacobucci,et al.  Brand diagnostics: Mapping branding effects using consumer associative networks , 1998, Eur. J. Oper. Res..

[42]  Bing Liu,et al.  Mining and summarizing customer reviews , 2004, KDD.

[43]  Andrew McCallum,et al.  Conditional Models of Identity Uncertainty with Application to Noun Coreference , 2004, NIPS.

[44]  Donald R. Lehmann,et al.  Judged Similarity and Brand-Switching Data as Similarity Measures , 1972 .

[45]  John R. Hauser,et al.  “Listening In” to Find and Explore New Combinations of Customer Needs , 2004 .

[46]  Frederick F. Reichheld,et al.  The Loyalty Effect: The Hidden Force Behind Growth, Profits, and Lasting Value (Эффект лояльности: скрытая движущая сила роста, прибыли и постоянной ценности) , 1996 .

[47]  R. Harshman,et al.  A Model for the Analysis of Asymmetric Data in Marketing Research , 1982 .

[48]  Allan D. Shocker,et al.  Customer-Oriented Approaches to Identifying Product-Markets , 1979 .

[49]  Qin He,et al.  Knowledge Discovery Through Co-Word Analysis , 1999, Libr. Trends.

[50]  Marti A. Hearst.  Automated Discovery of WordNet Relations , 2004 .

[51]  A. K. Mukhopadhyay.  Fuel Economy , 1918, Nature.

[52]  Jochen Dörre,et al.  Text mining: finding nuggets in mountains of textual data , 1999, KDD '99.

[53]  Neil R. Smalheiser,et al.  Information discovery from complementary literatures: Categorizing viruses as potential weapons , 2001, J. Assoc. Inf. Sci. Technol..

[54]  U. Simonsohn,et al.  Downloading Wisdom from Online Crowds , 2007 .

[55]  V. Srinivasan,et al.  A Simultaneous Approach to Market Segmentation and Market Structuring , 1987 .

[56]  Richard F. Yalch,et al.  Online Consumer Psychology : Understanding and Influencing Consumer Behavior in the Virtual World , 2005 .

[57]  Akihiro Inoue,et al.  Building Market Structures from Consumer Preferences , 1996 .

[58]  Ronen Feldman,et al.  Management's Tone Change, Post Earnings Announcement Drift and Accruals , 2009 .

[59]  Andrew McCallum,et al.  Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data , 2001, ICML.

[60]  Bing Liu,et al.  Opinion observer: analyzing and comparing opinions on the Web , 2005, WWW '05.

[61]  Allan Collins,et al.  A spreading-activation theory of semantic processing , 1975 .

[62]  Ronen Feldman,et al.  The Text Mining Handbook: Index , 2006 .

[63]  G. Bower,et al.  Human Associative Memory , 1973 .

[64]  Jacob Goldenberg,et al.  Extracting Product Comparisons from Discussion Boards , 2007, Seventh IEEE International Conference on Data Mining (ICDM 2007).

[65]  M E J Newman,et al.  Community structure in social and biological networks , 2001, Proceedings of the National Academy of Sciences of the United States of America.

[66]  Pradeep Chintagunta,et al.  The Effects of Online User Reviews on Movie Box Office Performance: Accounting for Sequential Rollout and Aggregation Across Local Markets , 2010, Mark. Sci..

[67]  Jon Duke,et al.  A quantitative analysis of adverse events and "overwarning" in drug labeling. , 2011, Archives of internal medicine.

[68]  M. Trusov,et al.  Estimating Aggregate Consumer Preferences from Online Product Reviews , 2010 .

[69]  Peeter W. J. Verlegh,et al.  The Firm's Management of Social Interactions , 2005 .

[70]  Mike Y. Chen,et al.  Yahoo! for Amazon: Sentiment Extraction from Small Talk on the Web , 2007, Manag. Sci..

[71]  Navot Akiva,et al.  Mining and Visualizing Online Web Content Using BAM: Brand Association Map , 2008, ICWSM.

[72]  Rob Malouf,et al.  Mining Web Text for Brand Associations , 2006, AAAI Spring Symposium: Computational Approaches to Analyzing Weblogs.

[73]  T. Newkirk.  Listening In , 1992 .