Welfare Properties of Profit Maximizing Recommender Systems: Theory and Results from a Randomized Experiment

Recommender systems have been introduced to help consumers navigate large sets of alternatives. They usually lead to more sales, which may increase consumer surplus and firm profit. In this paper, we ask whether firms may hurt consumers when they choose which recommender systems to use. We use data from a large scale field experiment ran using the video-on-demand system of a large telecommunications provider to measure the price elasticity of demand for movies placed in salient and non-salient slots on the TV screen. During this experiment, the firm randomized the slots in which movies were recommended to consumers as well as their prices. This setting readily allows for identifying the effects of price and slot on demand and thus compute consumer surplus. We find empirical evidence that consumers are less price elastic towards movies placed in salient slots. Using the outcomes of this experiment we simulate how consumer surplus and welfare change when the firm implements several recommender system, namely one that maximizes profit. We show that this system hurts both consumer surplus and welfare relative to the systems designed to maximize the latter. We also show that, at least in our setting, the system that maximizes profit does not generate less consumer surplus than some recommender systems often used in practice, such as content-based, lists of most sold, most rated and highest rated products. Yet, how much extra rent the firm can extract from strategically placing movies in salient slots is still a function of the popularity and quality of movies used to do so. Ultimately, our results question whether recommender systems embed mechanisms that extract excessive surplus from consumers, which may call for better scrutiny.

[1]  Gediminas Adomavicius,et al.  Toward the next generation of recommender systems: a survey of the state-of-the-art and possible extensions , 2005, IEEE Transactions on Knowledge and Data Engineering.

[2]  Ajay Kalra,et al.  The Impact of Advertising Positioning Strategies on Consumer Price Sensitivity , 1998 .

[3]  Matthew J. Salganik,et al.  Experimental Study of Inequality and Unpredictability in an Artificial Cultural Market , 2006, Science.

[4]  Dick R. Wittink,et al.  Empirical Generalizations About the Impact of Advertising on Price Sensitivity and Price , 1995 .

[5]  R. Thaler,et al.  Libertarian Paternalism , 2019, Encyclopedia of Law and Economics.

[6]  Dmitri Kuksov,et al.  When More Alternatives Lead to Less Choice , 2010, Mark. Sci..

[7]  S. Rosen The Economics of Superstars , 1981 .

[8]  Claire Mathieu,et al.  Maximizing profit using recommender systems , 2009, ArXiv.

[9]  David Heckerman,et al.  Empirical Analysis of Predictive Algorithms for Collaborative Filtering , 1998, UAI.

[10]  G. Kalyanaram,et al.  Nudge: Improving Decisions about Health, Wealth, and Happiness , 2011 .

[11]  G. Stigler The Economics of Information , 1961, Journal of Political Economy.

[12]  Matthew J. Salganik,et al.  Leading the Herd Astray: An Experimental Study of Self-fulfilling Prophecies in an Artificial Cultural Market , 2008, Social psychology quarterly.

[13]  Erik Brynjolfsson,et al.  Goodbye Pareto Principle, Hello Long Tail: The Effect of Search Costs on the Concentration of Product Sales , 2011, Manag. Sci..

[14]  R. Armstrong The Long Tail: Why the Future of Business Is Selling Less of More , 2008 .

[15]  Anita Elberse,et al.  Superstars and Underdogs: An Examination of the Long Tail Phenomenon in Video Sales , 2007 .

[16]  Dietmar Jannach,et al.  Price and Profit Awareness in Recommender Systems , 2017, ArXiv.

[17]  Juanjuan Zhang,et al.  Long Tail or Steep Tail? A Field Investigation into How Online Popularity Information Affects the Distribution of Customer Choices , 2007 .

[18]  Chun Liu,et al.  Social Influence Bias : A Randomized Experiment , 2014 .

[19]  John G. Lynch,et al.  Toward a Reconciliation of Market Power and Information Theories of Advertising Effects on Price Elasticity , 1995 .

[20]  Shawn P. Curley,et al.  Do Recommender Systems Manipulate Consumer Preferences? A Study of Anchoring Effects , 2013, Inf. Syst. Res..