Professional reviews as service: A mix method approach to assess the value of recommender systems in the entertainment industry

Abstract As a recommender system, Metacritic (metacritic.com) quantifies and aggregates reviews of entertainment products. The metascore published by Metacritic is controversial in its process for gathering, translating, and aggregating reviews. This study aims to assess the quality of Metacritic's scoring of movie critics’ reviews. By using a mixed methods approach integrating content analysis and multilevel analysis, we found that the movie review scores assigned by Metacritic reflecting the reviews’ semantic contents, being unbiased across the critics, and differentiating the movies reviewed. The paper provides, also, both theoretical implications, referred to an original framing of professional reviews in the service science, service systems and service quality; and practical implications related to the potential customers and for the entertainment industry.

[1]  Peter McKiernan,et al.  Synthesizing scenario planning and industry recipes through an analysis of the Hollywood film industry , 2020, Technological Forecasting and Social Change.

[2]  Steven M. Shugan,et al.  Film Critics: Influencers or Predictors? , 1997 .

[3]  Kaiqiang Guo,et al.  The impact of online reviews on exhibitor behaviour: evidence from movie industry , 2017, Enterp. Inf. Syst..

[4]  Joseph Price,et al.  Ratings and Revenues: Evidence from Movie Ratings , 2013 .

[5]  Paul P. Maglio,et al.  Fundamentals of service science , 2008 .

[6]  Li Xiang-yang,et al.  How successful movies affect performance of sequels: Signal theory and brand extension theory in motion picture industry , 2013, 2013 International Conference on Management Science and Engineering 20th Annual Conference Proceedings.

[7]  S. Ravid.  Information, Blockbusters and Stars? A Study of the Film Industry , 1997 .

[8]  Dai Yao,et al.  Spillover Effects in Seeded Word-of-Mouth Marketing Campaigns , 2016, Mark. Sci..

[9]  Muhammad Ibrahim,et al.  Design and Application of a Multi-Variant Expert System Using Apache Hadoop Framework , 2018, Sustainability.

[10]  Charles B. Weinberg,et al.  Silverscreener: a Modeling Approach to Movie Screens Management , 1999 .

[11]  David J. Miller,et al.  The Resource-Based View of the Firm in Two Environments: The Hollywood Film Studios From 1936 to 1965 , 1996 .

[12]  Thomas Solon Simonet.  Regression analysis of prior experiences of key production personnel as predictors of revenues from high-grossing motion pictures in American release , 1980 .

[13]  Robin D. Burke,et al.  Hybrid Web Recommender Systems , 2007, The Adaptive Web.

[14]  Zhigeng Fang,et al.  Editorial: towards service science, engineering and practice , 2007 .

[15]  X. Zhang,et al.  Impact of Online Consumer Reviews on Sales: The Moderating Role of Product and Consumer Characteristics , 2010 .

[16]  Stefano Bresciani,et al.  Collaborative modes with Cultural and Creative Industries and innovation performance: The moderating role of heterogeneous sources of knowledge and absorptive capacity , 2020 .

[17]  Junghoon Moon,et al.  Exploring the effect of e-WOM participation on e-Loyalty in e-commerce , 2013, Decis. Support Syst..

[18]  P. Hamilton,et al.  Off-Hollywood: The Making and Marketing of Independent Films , 1990 .

[19]  A. McKee.  Entertainment and Fun , 2016 .

[20]  Richard T. Gretz,et al.  Is everybody an expert? An investigation into the impact of professional versus user reviews on movie revenues , 2020, Journal of Cultural Economics.

[21]  Mark Baimbridge,et al.  Movie admissions and rental income: the case of James Bond , 1997 .

[22]  Anselm L. Strauss,et al.  Basics of qualitative research : techniques and procedures for developing grounded theory , 1998 .

[23]  Mimmo Parente,et al.  An ontology-driven context-aware recommender system for indoor shopping based on cellular automata , 2017, J. Ambient Intell. Humaniz. Comput..

[24]  Ya-Ling Chiu,et al.  The impact of online movie word-of-mouth on consumer choice , 2019, International Marketing Review.

[25]  Elias G. Carayannis,et al.  Culture and Cooperative Strategies: Knowledge Management Perspectives , 2012 .

[26]  Suman Basuroy,et al.  Distributors and film critics: does it take two to Tango? , 2006 .

[27]  Christian Burgers,et al.  Do consumer critics write differently from professional critics? A genre analysis of online film reviews , 2013 .

[28]  Barry Litman,et al.  Predicting financial success of motion pictures: The '80s experience , 1989 .

[29]  J. Eliashberg,et al.  MOVIEMOD: An Implementable Decision-Support System for Prerelease Market Evaluation of Motion Pictures , 2000 .

[30]  Luca Vincenzo Ballestra,et al.  Harvesting reflective knowledge exchange for inbound open innovation in complex collaborative networks: an empirical verification in Europe , 2020, J. Knowl. Manag..

[31]  Francesco Polese,et al.  Reflections on Service Systems Boundaries: A Viable Systems Perspective – The Case of the London Borough of Sutton , 2012 .

[32]  Harvey Goldstein,et al.  Cross-classified and multiple membership structures in multilevel models : an introduction and review , 2006 .

[33]  Hal R. Varian,et al.  Information rules - a strategic guide to the network economy , 1999 .

[34]  Manlio Del Giudice,et al.  How are decision systems changing? The contribution of social media to the management of decisional liquefaction , 2016, J. Decis. Syst..

[35]  Natalia Kryvinska,et al.  Service Systems and Service Innovation: Two Pillars of Service Science , 2016, ANT/SEIT.

[36]  Hsin Hsin Chang,et al.  An examination of negative e-WOM adoption: Brand commitment as a moderator , 2014, Decis. Support Syst..

[37]  Jens Mueller,et al.  Improving innovation performance through knowledge acquisition: the moderating role of employee retention and human resource management practices , 2018, J. Knowl. Manag..

[38]  Byeng-Hee Chang,et al.  Devising a Practical Model for Predicting Theatrical Movie Success: Focusing on the Experience Good Property , 2005 .

[39]  Gorham Anders Kindem,et al.  The American movie industry : the business of motion pictures , 1982 .

[40]  Francesco Caputo,et al.  Determinants for Value Cocreation and Collaborative Paths in Complex Service Systems: A Focus on (Smart) Cities , 2018, Service Science.

[41]  Rick W. Busselle,et al.  Fictionality and Perceived Realism in Experiencing Stories: A Model of Narrative Comprehension and Engagement , 2008 .

[42]  Wenqi Zhou,et al.  Online product rating manipulation and market performance , 2015, Computer.

[43]  Ching-Fu Chen,et al.  Experience quality, perceived value, satisfaction and behavioral intentions for heritage tourists , 2010 .

[44]  Bill Buckles,et al.  Movie success prediction using data mining , 2017, 2017 8th International Conference on Computing, Communication and Networking Technologies (ICCCNT).

[45]  Stefano Bresciani,et al.  Shifting Intra‐ and Inter‐Organizational Innovation Processes Towards Digital Business: An Empirical Analysis of SMEs , 2017 .

[46]  Adams Greenwood-Ericksen,et al.  On the Validity of Metacritic in Assessing Game Value , 2013, Eludamos: Journal for Computer Game Culture.

[47]  Samer Faraj,et al.  Why Should I Share? Examining Social Capital and Knowledge Contribution in Electronic Networks of Practice , 2005, MIS Q..

[48]  Harvey Goldstein,et al.  Multiple membership multiple classification (MMMC) models , 2001 .

[49]  Jay Prag,et al.  An empirical study of the determinants of revenues and marketing expenditures in the motion picture industry , 1994 .

[50]  Andrea Groeppel-Klein,et al.  Pretty Woman Or Erin Brockovich? Unconscious and Conscious Reactions to Commercials and Movies Shaped By Fairy Tale Archetpyes – Results From Two Experimental Studies , 2006 .

[51]  Gerda Gemser,et al.  The impact of film reviews on the box office performance of art house versus mainstream motion pictures , 2007 .

[52]  Lutz Kolbe,et al.  Knowledge-enabled customer relationship management: integrating customer relationship management and knowledge management concepts[1] , 2003, J. Knowl. Manag..

[53]  Wei-Jaw Deng,et al.  The relationships among service quality, perceived value, customer satisfaction, and post-purchase intention in mobile value-added services , 2009, Comput. Hum. Behav..

[54]  Christy Collis,et al.  Entertainment industries: Entertainment as a cultural system , 2012 .

[55]  Anthony S. Bryk,et al.  Hierarchical Linear Models: Applications and Data Analysis Methods , 1992 .

[56]  A. Fagerstrøm.  The Motivating Effect of Antecedent Stimuli on the Web Shop: A Conjoint Analysis of the Impact of Antecedent Stimuli at the Point of Online Purchase , 2010 .

[57]  Peter Vorderer,et al.  Does Entertainment Suffer From Interactivity? The Impact of Watching an Interactive TV Movie on Viewers' Experience of Entertainment , 2001 .

[58]  Stephen L. Vargo,et al.  Toward a conceptual foundation for service science: Contributions from service-dominant logic , 2008, IBM Syst. J..

[59]  Alexeis Garcia-Perez,et al.  A knowledge-based view of people and technology: directions for a value co-creation-based learning organisation , 2019, J. Knowl. Manag..

[60]  Albert N. Greco.  The market for consumer books in the U.S.: 1985–1995 , 1997 .

[61]  Tianjie Deng.  Investigating the effects of textual reviews from consumers and critics on movie sales , 2020, Online Inf. Rev..

[62]  M. Holbrook.  Consumer Value: A Framework for Analysis and Research , 1999 .

[63]  J. Alba,et al.  The Effects of Frequency Knowledge On Consumer Decision Making , 1987 .

[64]  Measuring Service Quality: A Reexamination and Extension , 1992 .

[65]  Yong Liu,et al.  When do Third-Party Product Reviews Affect Firm Value and what can Firms Do? The Case of Media Critics and Professional Movie Reviews , 2012 .

[66]  Johnny Saldaña,et al.  The Coding Manual for Qualitative Researchers , 2009 .

[67]  Srinivas K. Reddy,et al.  Exploring the Determinants of Broadway Show Success , 1998 .

[68]  Narasimhan Jegadeesh,et al.  Analyzing the Analysts: When Do Recommendations Add Value? , 2002 .

[69]  Maria Vincenza Ciasullo,et al.  Service Innovations in the Healthcare Service Ecosystem: A Case Study , 2017, Syst..

[70]  Paul P. Maglio,et al.  The service system is the basic abstraction of service science , 2009, Inf. Syst. E Bus. Manag..

[71]  Tammo H. A. Bijmolt,et al.  The Effect of Electronic Word of Mouth on Sales: A Meta-Analytic Review of Platform, Product, and Metric Factors , 2016 .

[72]  Lorin M. Hitt,et al.  Self Selection and Information Role of Online Product Reviews , 2007, Inf. Syst. Res..

[73]  Christopher P. Furner,et al.  Electronic word-of-mouth and information overload in an experiential service industry , 2016 .

[74]  Jonah Berger,et al.  Positive Effects of Negative Publicity: When Negative Reviews Increase Sales , 2009, Mark. Sci..

[75]  W. Walls,et al.  Uncertainty in the Movie Industry: Does Star Power Reduce the Terror of the Box Office? , 1999 .

[76]  Dimple R. Thadani,et al.  The impact of electronic word-of-mouth communication: A literature analysis and integrative model , 2012, Decis. Support Syst..

[77]  Wen-Chin Tsao,et al.  Which type of online review is more persuasive? The influence of consumer reviews and critic ratings on moviegoers , 2014, Electron. Commer. Res..

[78]  L. Becchetti,et al.  The Determinants of Motion Picture Box Office Performance: Evidence from Movies Produced in Italy , 1999 .

[79]  Raffaele Filieri.  What makes online reviews helpful? A diagnosticity-adoption framework to explain informational and normative influences in e-WOM , 2015 .

[80]  J. J. Cronin,et al.  Assessing the effects of quality, value, and customer satisfaction on consumer behavioral intentions in service environments , 2000 .

[81]  Mónica Cortiñas,et al.  The impact of expert opinion in consumer perception of wines , 2013 .

[82]  Dwayne D. Gremler,et al.  Electronic word-of-mouth via consumer-opinion platforms: What motivates consumers to articulate themselves on the Internet? , 2004 .

[83]  Yong Liu.  Word of Mouth for Movies: Its Dynamics and Impact on Box Office Revenue , 2006 .

[84]  B. Rooney,et al.  Actually in the Cinema: A Field Study Comparing Real 3D and 2D Movie Patrons' Attention, Emotion, and Film Satisfaction , 2013 .

[85]  P. Mitran,et al.  Consumers’ Decision-Making Process on Social Commerce Platforms: Online Trust, Perceived Risk, and Purchase Intentions , 2020, Frontiers in Psychology.

[86]  Fan Li,et al.  Scores vs. stars: A regression discontinuity study of online consumer reviews , 2019, Inf. Manag..

[87]  Angelo Bonfanti,et al.  Italian Craft Firms Between Digital Manufacturing, Open Innovation, and Servitization , 2018 .

[88]  Kristine de Valck,et al.  Conceptualizing the electronic word-of-mouth process: What we know and need to know about eWOM creation, exposure, and evaluation , 2020 .

[89]  D. Sarno,et al.  The Influence of Cognitive Dimensions on the Consumer-SME Relationship: A Sustainability-Oriented View , 2018, Sustainability.

[90]  J. W. Hutchinson,et al.  Knowledge Calibration: What Consumers Know and What They Think They Know , 2000 .

[91]  Wenjing Duan,et al.  Do Professional Reviews Affect Online User Choices Through User Reviews? An Empirical Study , 2016 .

[92]  H. Goldstein.  Multilevel Statistical Models , 2006 .

[93]  Dean Keith Simonton.  Cinematic creativity and production budgets: Does money make the movie? , 2005 .

[94]  Petter Gottschalk.  Corporate Social Responsibility, Governance and Corporate Reputation , 2011 .

[95]  Ma’moun A. Habiballah,et al.  A model of service quality aspects conveyed in hotel advertising , 2017 .

[96]  Thomas H. Davenport,et al.  Book review:Working knowledge: How organizations manage what they know. Thomas H. Davenport and Laurence Prusak. Harvard Business School Press, 1998. $29.95US. ISBN 0‐87584‐655‐6 , 1998 .

[97]  Carol C. Bienstock,et al.  Measuring Service Quality in E-Retailing , 2006 .

[98]  Jialie Chen,et al.  The impact of advertising content on movie revenues , 2017 .

[99]  H. Goldstein,et al.  Efficient Analysis of Mixed Hierarchical and Cross-Classified Random Structures Using a Multilevel Model , 1994 .

[100]  Christopher M. Snyder,et al.  The Influence of Expert Reviews on Consumer Demand for Experience Goods: A Case Study of Movie Critics , 2005 .

[101]  Francesco Polese,et al.  Smart Service Systems and Viable Service Systems: Applying Systems Theory to Service Science , 2010 .

[102]  Monic Sun,et al.  How Does the Variance of Product Ratings Matter? , 2010, Manag. Sci..

[103]  Pradeep Kumar Ponnamma Divakaran.  Technology-enabled community data for gaining pre-release brand insights , 2018 .

[104]  Rakhi Thakur.  Customer engagement and online reviews , 2018 .

[105]  Ingoo Han,et al.  The Effect of On-Line Consumer Reviews on Consumer Purchasing Intention: The Moderating Role of Involvement , 2007, Int. J. Electron. Commer..

[106]  Catherine A. Cole,et al.  Consumer decision making and aging: Current knowledge and future directions , 2009 .

[107]  Christopher P. Holland,et al.  The impact of consumer archetypes on online purchase decision-making processes and outcomes: A behavioural process perspective , 2018, Journal of Business Research.

[108]  Suman Basuroy,et al.  How Critical are Critical Reviews? The Box Office Effects of Film Critics, Star Power, and Budgets , 2003 .

[109]  Phillip C. Stocken,et al.  An Analysis of Stock Recommendations , 1998 .

[110]  R. Yin.  Case Study Research: Design and Methods , 1984 .

[111]  Lisa Petrides,et al.  Knowledge Management for School Leaders: An Ecological Framework for Thinking Schools , 2002, Teachers College Record: The Voice of Scholarship in Education.

[112]  Roni Michaely,et al.  Conflict of interest and the credibility of underwriter analyst recommendations , 1999 .

[113]  Roberto Bruni,et al.  Enabling actors' viable behaviour: reflections upon the link between viability and complexity within smart service system , 2018 .

[114]  Robert K. Yin,et al.  Case Study Research and Applications: Design and Methods , 2017 .