Decision-based model selection

Abstract A key step in data-driven decision making is the choice of a suitable mathematical model. Complex models that give an accurate description of reality may depend on many parameters that are difficult to estimate; in addition, the optimization problem corresponding to such models may be computationally intractable and only approximately solvable. Simple models with only a few unknown parameters may be misspecified, but also easier to estimate and optimize. With such different models and some initial data at hand, a decision maker would want to know which model produces the best decisions. In this paper we propose a decision-based model-selection method that addresses this question.

[1]  C. Holmes,et al.  Approximate Models and Robust Decisions , 2014, 1402.6118.

[2]  Anton J. Kleywegt,et al.  Models of the Spiral-Down Effect in Revenue Management , 2006, Oper. Res..

[3]  H. Akaike,et al.  Information Theory and an Extension of the Maximum Likelihood Principle , 1973 .

[4]  N. Hjort,et al.  The Focused Information Criterion , 2003 .

[5]  C. L. Mallows.  Some comments on C_p , 1973 .

[6]  R. Tibshirani.  Regression Shrinkage and Selection via the Lasso , 1996 .

[7]  Anton J. Kleywegt,et al.  Learning and Pricing with Models That Do Not Explicitly Incorporate Competition , 2015, Oper. Res..

[8]  M. Stone.  Cross‐Validatory Choice and Assessment of Statistical Predictions , 1976 .

[9]  Andrew E. B. Lim,et al.  Model Uncertainty, Robust Optimization and Learning , 2006 .

[10]  Isabelle Guyon,et al.  Model Selection: Beyond the Bayesian/Frequentist Divide , 2010, J. Mach. Learn. Res..

[11]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[12]  Mohsen Bayati,et al.  Online Decision Making with High-Dimensional Covariates , 2020, Oper. Res..

[13]  Thijs van Ommen,et al.  Inconsistency of Bayesian Inference for Misspecified Linear Models, and a Proposal for Repairing It , 2014, 1412.3730.

[14]  Wasserman,et al.  Bayesian Model Selection and Model Averaging. , 2000, Journal of mathematical psychology.

[15]  Yi-Hao Kao,et al.  Directed Regression , 2009, NIPS.

[16]  J. Rissanen,et al.  Modeling By Shortest Data Description* , 1978, Autom..

[17]  Richard Ratliff,et al.  A General Attraction Model and Sales-Based Linear Program for Network Revenue Management Under Customer Choice , 2015, Oper. Res..

[18]  Seymour Geisser,et al.  The Predictive Sample Reuse Method with Applications , 1975 .

[19]  Trevor Hastie,et al.  The Elements of Statistical Learning , 2001 .

[20]  Gábor Lugosi,et al.  Introduction to Statistical Learning Theory , 2004, Advanced Lectures on Machine Learning.

[21]  Omar Besbes,et al.  On the (Surprising) Sufficiency of Linear Models for Dynamic Pricing with Demand Learning , 2014, Manag. Sci..

[22]  Gérard P. Cachon,et al.  Implementation of the Newsvendor Model with Clearance Pricing: How to (and How Not to) Estimate a Salvage Value , 2007, Manuf. Serv. Oper. Manag..

[23]  Sylvain Arlot,et al.  A survey of cross-validation procedures for model selection , 2009, 0907.4728.

[24]  H. Jeffreys,et al.  Theory of probability , 1896 .

[25]  Zucchini,et al.  An Introduction to Model Selection. , 2000, Journal of mathematical psychology.

[26]  Walter Zucchini,et al.  Model Selection , 2011, International Encyclopedia of Statistical Science.

[27]  Robert C. Nickerson,et al.  The Use and Value of Models in Decision Analysis , 1980, Oper. Res..

[28]  Yi-Hao Kao,et al.  Directed Principal Component Analysis , 2014, Oper. Res..

[29]  Soonhui Lee,et al.  Newsvendor-type models with decision-dependent uncertainty , 2012, Math. Methods Oper. Res..

[30]  William L. Cooper,et al.  On the Use of Buy Up as a Model of Customer Choice in Revenue Management , 2012 .

[31]  P. Grünwald.  The Minimum Description Length Principle (Adaptive Computation and Machine Learning) , 2007 .

[32]  J. George Shanthikumar,et al.  A practical inventory control policy using operational statistics , 2005, Oper. Res. Lett..

[33]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[34]  Yi-Hao Kao,et al.  Learning a factor model via regularized PCA , 2011, Machine Learning.

[35]  J. George Shanthikumar,et al.  Solving operational statistics via a Bayesian analysis , 2008, Oper. Res. Lett..

[36]  G. Schwarz.  Estimating the Dimension of a Model , 1978 .

[37]  Robert Phillips,et al.  Testing the Validity of a Demand Model: An Operations Perspective , 2010, Manuf. Serv. Oper. Manag..

[38]  David R. Anderson,et al.  Model selection and multimodel inference : a practical information-theoretic approach , 2003 .

[39]  Colin L. Mallows,et al.  Some Comments on Cp , 2000, Technometrics.

[40]  J. George Shanthikumar,et al.  Inventory Policy with Parametric Demand: Operational Statistics, Linear Correction, and Regression , 2012 .

[41]  Nils Lid Hjort,et al.  Model Selection and Model Averaging , 2001 .

[42]  Bradley P. Carlin,et al.  Bayesian measures of model complexity and fit , 2002 .

[43]  H. Jeffreys.  Some Tests of Significance, Treated by the Theory of Probability , 1935, Mathematical Proceedings of the Cambridge Philosophical Society.