The Value of Field Experiments

The feasibility of using field experiments to optimize marketing decisions remains relatively unstudied. We investigate category pricing decisions that require estimating a large matrix of cross-product demand elasticities and ask the following question: How many experiments are required as the number of products in the category grows? Our main result demonstrates that if the categories have a favorable structure, we can learn faster and reduce the number of experiments that are required: the number of experiments required may grow just logarithmically with the number of products. These findings potentially have important implications for the application of field experiments. Firms may be able to obtain meaningful estimates using a practically feasible number of experiments, even in categories with a large number of products. We also provide a relatively simple mechanism that firms can use to evaluate whether a category has a structure that makes it feasible to use field experiments to set prices. We illustrate how to accomplish this using either a sample of historical data or a pilot set of experiments. We also discuss how to evaluate whether field experiments can help optimize other marketing decisions, such as selecting which products to advertise or promote.Data, as supplemental material, are available at http://dx.doi.org/10.1287/mnsc.2014.2066 . This paper was accepted by Pradeep Chintagunta, marketing .

[1]  A. Tversky Elimination by aspects: A theory of choice. , 1972 .

[2]  J. Burningham The shopping basket , 1980 .

[3]  Leslie G. Valiant,et al.  A theory of the learnable , 1984, STOC '84.

[4]  J. Sengupta The Nonparametric Approach , 1989 .

[5]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[6]  E. George,et al.  Journal of the American Statistical Association is currently published by American Statistical Association. , 2007 .

[7]  S. D. Chatterji Proceedings of the International Congress of Mathematicians , 1995 .

[8]  Tülin Erdem A Dynamic Analysis of Market Structure Based on Panel Data , 1996 .

[9]  R. Tibshirani Regression Shrinkage and Selection via the Lasso , 1996 .

[10]  Joel Huber,et al.  The Importance of Utility Balance in Efficient Choice Designs , 1996 .

[11]  M. Keane Modeling Heterogeneity and State Dependence in Consumer Choice Behavior , 1997 .

[12]  Sunil Gupta,et al.  The Shopping Basket: A Model for Multicategory Purchase Incidence Decisions , 1999 .

[13]  J. M. Villas-Boas,et al.  Endogeneity in Brand Choice Models , 1999 .

[14]  Pradeep K. Chintagunta,et al.  Investigating Household State Dependence Effects across Categories , 1999 .

[15]  D. Hensher,et al.  Stated Choice Methods: Analysis and Applications , 2000 .

[16]  Eric T. Anderson,et al.  Are Sale Signs Less Effective When More Products Have Them , 2001 .

[17]  John R. Hauser,et al.  Fast Polyhedral Adaptive Conjoint Estimation , 2002 .

[18]  M. Wedel,et al.  Designing Conjoint Choice Experiments Using Managers' Prior Beliefs , 2001 .

[19]  Jordan J. Louviere,et al.  A 20+ Years’ Retrospective on Choice Experiments , 2004 .

[20]  Cristian Darie,et al.  The Shopping Basket , 2004 .

[21]  Emmanuel J. Candès,et al.  Decoding by linear programming , 2005, IEEE Transactions on Information Theory.

[22]  Emmanuel J. Candès,et al.  Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information , 2004, IEEE Transactions on Information Theory.

[23]  E.J. Candes Compressive Sampling , 2022 .

[24]  Thomas P. Hayes,et al.  Stochastic Linear Optimization under Bandit Feedback , 2008, COLT.

[25]  Peter E. Rossi,et al.  Category Pricing with State-Dependent Utility , 2008, Mark. Sci..

[26]  John N. Tsitsiklis,et al.  A Structured Multiarmed Bandit Problem and the Greedy Policy , 2008, IEEE Transactions on Automatic Control.

[27]  Martin J. Wainwright,et al.  Information-theoretic limits on sparsity recovery in the high-dimensional and noisy setting , 2009, IEEE Trans. Inf. Theory.

[28]  Martin J. Wainwright,et al.  Information-Theoretic Limits on Sparsity Recovery in the High-Dimensional and Noisy Setting , 2007, IEEE Transactions on Information Theory.

[29]  T. Evgeniou,et al.  Disjunctions of Conjunctions, Cognitive Simplicity, and Consideration Sets , 2010 .

[30]  Qing Liu,et al.  Efficient Choice Designs for a Consider-Then-Choose Model , 2011, Mark. Sci..

[31]  Devavrat Shah,et al.  A Nonparametric Approach to Modeling Choice with Limited Data , 2009, Manag. Sci..

[32]  Eric T. Anderson,et al.  Advertising in a Competitive Market: The Role of Product Standards, Customer Learning, and Switching Costs , 2013 .

[33]  Qiuyuan Jimmy Li The value of field experiments in estimating demand elasticities and maximizing profit , 2014 .