Residuals-based distributionally robust optimization with covariate information.

We consider data-driven approaches that integrate a machine learning prediction model within distributionally robust optimization (DRO) given limited joint observations of uncertain parameters and covariates. Our framework is flexible in the sense that it can accommodate a variety of learning setups and DRO ambiguity sets. We investigate the asymptotic and finite sample properties of solutions obtained using Wasserstein, sample robust optimization, and phi-divergence-based ambiguity sets within our DRO formulations, and explore cross-validation approaches for sizing these ambiguity sets. Through numerical experiments, we validate our theoretical results, study the effectiveness of our approaches for sizing ambiguity sets, and illustrate the benefits of our DRO formulations in the limited data regime even when the prediction model is misspecified.

[1]  Brad Sturt A Data-Driven Approach for Multi-Stage Linear Optimization , 2019 .

[2]  Sanjay Mehrotra,et al.  Distributionally Robust Optimization: A Review , 2019, ArXiv.

[3]  David P. Morton,et al.  Monte Carlo bounding techniques for determining solution quality in stochastic programs , 1999, Oper. Res. Lett..

[4]  Alexander Shapiro,et al.  Lectures on Stochastic Programming: Modeling and Theory , 2009 .

[5]  Rui Gao Finite-Sample Guarantees for Wasserstein Distributionally Robust Optimization: Breaking the Curse of Dimensionality , 2020, ArXiv.

[6]  Tito Homem-de-Mello,et al.  Monte Carlo sampling-based methods for stochastic optimization , 2014 .

[7]  C. Villani Optimal Transport: Old and New , 2008 .

[8]  Henry Lam,et al.  Recovering Best Statistical Guarantees via the Empirical Divergence-Based Distributionally Robust Optimization , 2016, Oper. Res..

[9]  Daniel Kuhn,et al.  Robust Data-Driven Dynamic Programming , 2013, NIPS.

[10]  Fan Zhang,et al.  Distributionally Robust Local Non-parametric Conditional Estimation , 2020, NeurIPS.

[11]  Juan M. Morales,et al.  Distributionally robust stochastic programs with side information based on trimmings , 2020 .

[12]  M. KarthyekRajhaaA.,et al.  Robust Wasserstein profile inference and applications to machine learning , 2019, J. Appl. Probab..

[13]  Vishal Gupta,et al.  Robust sample average approximation , 2014, Math. Program..

[14]  R. Rockafellar,et al.  Optimization of conditional value-at risk , 2000 .

[15]  Weijun Xie,et al.  Tractable reformulations of two-stage distributionally robust linear programs over the type-∞ Wasserstein ball , 2020, Oper. Res. Lett..

[16]  Xi Chen,et al.  Wasserstein Distributional Robustness and Regularization in Statistical Learning , 2017, ArXiv.

[17]  Sanjay Mehrotra,et al.  Decomposition Algorithms for Two-Stage Distributionally Robust Mixed Binary Programs , 2018, SIAM J. Optim..

[18]  Alan Edelman,et al.  Julia: A Fresh Approach to Numerical Computing , 2014, SIAM Rev..

[19]  Dimitris Bertsimas,et al.  From Predictive to Prescriptive Analytics , 2014, Manag. Sci..

[20]  S. Sen,et al.  Learning Enabled Optimization : Towards a Fusion of Statistical Learning and Stochastic Programming , 2018 .

[21]  Mihai Anitescu,et al.  Distributionally Robust Optimization with Correlated Data from Vector Autoregressive Processes , 2019, Oper. Res. Lett..

[22]  Trevor Hastie,et al.  Regularization Paths for Generalized Linear Models via Coordinate Descent. , 2010, Journal of statistical software.

[23]  Dimitris Bertsimas,et al.  Two-stage sample robust optimization , 2019, 1907.07142.

[24]  Jon A. Wellner,et al.  Weak Convergence and Empirical Processes: With Applications to Statistics , 1996 .

[25]  James R. Luedtke,et al.  Data-Driven Sample Average Approximation with Covariate Information , 2022, 2207.13554.

[26]  John Duchi,et al.  Statistics of Robust Optimization: A Generalized Empirical Likelihood Approach , 2016, Math. Oper. Res..

[27]  Jérémie Gallien,et al.  Dynamic Procurement of New Products with Covariate Information: The Residual Tree Method , 2019, Manuf. Serv. Oper. Manag..

[28]  Henry Lam,et al.  Robust Sensitivity Analysis for Stochastic Systems , 2013, Math. Oper. Res..

[29]  G. Pflug,et al.  Ambiguity in portfolio selection , 2007 .

[30]  A. Guillin,et al.  On the rate of convergence in Wasserstein distance of the empirical measure , 2013, 1312.2128.

[31]  Erick Delage,et al.  Generalization bounds for regularized portfolio selection with market side information , 2018, INFOR Inf. Syst. Oper. Res..

[32]  Cynthia Rudin,et al.  The Big Data Newsvendor: Practical Insights from Machine Learning , 2013, Oper. Res..

[33]  Anja De Waegenaere,et al.  Robust Solutions of Optimization Problems Affected by Uncertain Probabilities , 2011, Manag. Sci..

[34]  Karthyek R. A. Murthy,et al.  Confidence Regions in Wasserstein Distributionally Robust Estimation , 2019, Biometrika.

[35]  Dimitris Bertsimas,et al.  Dynamic optimization with side information , 2019, Eur. J. Oper. Res..

[36]  Shie Mannor,et al.  A distributional interpretation of robust optimization , 2010, 2010 48th Annual Allerton Conference on Communication, Control, and Computing (Allerton).

[37]  Dorota Kurowicka,et al.  Generating random correlation matrices based on vines and extended onion method , 2009, J. Multivar. Anal..

[38]  Viet Anh Nguyen,et al.  Wasserstein Distributionally Robust Optimization: Theory and Applications in Machine Learning , 2019, Operations Research & Management Science in the Age of Analytics.

[39]  Bart P. G. Van Parys,et al.  Bootstrap robust prescriptive analytics , 2017, Mathematical Programming.

[40]  Daniel Kuhn,et al.  Conic Programming Reformulations of Two-Stage Distributionally Robust Linear Programs over Wasserstein Balls , 2016, Oper. Res..

[41]  Güzin Bayraksan,et al.  Data-Driven Stochastic Programming Using Phi-Divergences , 2015 .

[42]  A. Kleywegt,et al.  Distributionally Robust Stochastic Optimization with Wasserstein Distance , 2016, Math. Oper. Res..

[43]  Daniel Kuhn,et al.  Data-driven distributionally robust optimization using the Wasserstein metric: performance guarantees and tractable reformulations , 2015, Mathematical Programming.

[44]  Iain Dunning,et al.  JuMP: A Modeling Language for Mathematical Optimization , 2015, SIAM Rev..

[45]  Nicolás García Trillos,et al.  On the rate of convergence of empirical measures in $\infty$-transportation distance , 2014, 1407.1157.