Sample Average Approximation Method for Compound Stochastic Optimization Problems

The paper studies stochastic optimization (programming) problems with compound functions containing expectations and extreme values of other random functions as arguments. Compound functions arise in various applications. A typical example is a variance function of nonlinear outcomes. Other examples include stochastic minimax problems, econometric models with latent variables, and multilevel and multicriteria stochastic optimization problems. As a solution technique a sample average approximation (SAA) method (also known as statistical or empirical (sample) mean method) is used. The method consists in approximation of all expectation functions by their empirical means and solving the resulting approximate deterministic optimization problems. In stochastic optimization, this method is widely used for optimization of standard expectation functions under constraints. In this paper, SAA method is extended to general compound stochastic optimization problems. The conditions for convergence in mean, almost sure...

[1]  A. Shapiro,et al.  Stochastic mathematical programs with equilibrium constraints, modelling and sample average approximation , 2008 .

[2]  W. Hoeffding.  Probability Inequalities for sums of Bounded Random Variables , 1963 .

[3]  R. Wets,et al.  Epi‐consistency of convex stochastic programs , 1991 .

[4]  Colin McDiarmid,et al.  Surveys in Combinatorics, 1989: On the method of bounded differences , 1989 .

[5]  Normalized convergence of random variables , 1992 .

[6]  Pavel S. Knopov,et al.  Empirical Estimates in Stochastic Optimization and Identification , 2002 .

[7]  Yuri M. Ermoliev.  Stochastic Quasigradient Methods , 2009, Encyclopedia of Optimization.

[8]  R. Tyrrell Rockafellar,et al.  Asymptotic Theory for Solutions in Statistical Estimation and Stochastic Programming , 1993, Math. Oper. Res..

[9]  Huifu Xu,et al.  A note on uniform exponential convergence of sample average approximation of random functions , 2012 .

[10]  Daniel Ralph,et al.  Convergence of Stationary Points of Sample Average Two-Stage Stochastic Programs: A Generalized Equation Approach , 2011, Math. Oper. Res..

[11]  J. Dupacová.  The minimax approach to stochastic programming and an illustrative application , 1987 .

[12]  Jon A. Wellner,et al.  Weak Convergence and Empirical Processes: With Applications to Statistics , 1996 .

[13]  Tito Homem-de-Mello,et al.  On Rates of Convergence for Stochastic Optimization Problems Under Non--Independent and Identically Distributed Sampling , 2008, SIAM J. Optim..

[14]  Yuri M. Ermoliev,et al.  Stochastic Quasigradient Methods in Minimax Problems , 2009, Encyclopedia of Optimization.

[15]  Normalized convergence of random variables and its applications , 1990 .

[16]  Y. Ermoliev,et al.  Stochastic Optimization Problems with Incomplete Information on Distribution Functions , 1985 .

[17]  Alexander Shapiro,et al.  On a Class of Minimax Stochastic Programs , 2004, SIAM J. Optim..

[18]  Huifu Xu.  Uniform exponential convergence of sample average random functions under general sampling with applications in stochastic programming , 2010 .

[19]  J. Shanthikumar,et al.  Multivariate Stochastic Orders , 2007 .

[20]  A. Shapiro.  Asymptotic Properties of Statistical Estimators in Stochastic Programming , 1989 .

[21]  Yuri Ermoliev,et al.  Numerical techniques for stochastic optimization , 1988 .

[22]  R. Wets,et al.  Consistency of Minimizers and the SLLN for Stochastic Programs 1 , 1995 .

[23]  Felipe Cucker,et al.  On the mathematical foundations of learning , 2001 .

[24]  M. Talagrand.  Sharper Bounds for Gaussian and Empirical Processes , 1994 .

[25]  Y. Ermoliev,et al.  Normalized convergence in stochastic optimization , 1991, Ann. Oper. Res..

[26]  Y. Ermoliev,et al.  Systemic Risk and Security Management , 2012 .

[27]  V. I. Norkin,et al.  Convergence of the empirical mean method in statistics and stochastic programming , 1992 .

[28]  Huifu Xu,et al.  Monte Carlo methods for mean-risk optimization and portfolio selection , 2012, Comput. Manag. Sci..

[29]  Yuri Ermoliev,et al.  Stochastic quasigradient methods. Numerical techniques for stochastic optimization , 1988 .

[30]  Alexei A. Gaivoronski,et al.  A numerical method for solving stochastic programming problems with moment constraints on a distribution function , 1991, Ann. Oper. Res..

[31]  R. Wets,et al.  Stochastic programming , 1989 .

[32]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[33]  Vogel Silvia.  Qualitative stability of stochastic programs with applications in asymptotic statistics , 2005 .

[34]  Roger J.-B. Wets,et al.  Statistical estimation from an optimization viewpoint , 1999, Ann. Oper. Res..

[35]  Christian Gourieroux,et al.  Simulation-based econometric methods , 1996 .

[36]  Felipe Cucker,et al.  Learning Theory: An Approximation Theory Viewpoint: Index , 2007 .

[37]  Alexander Shapiro,et al.  Stochastic Approximation approach to Stochastic Programming , 2013 .

[38]  M. Talagrand,et al.  Probability in Banach Spaces: Isoperimetry and Processes , 1991 .

[39]  W. Sharpe,et al.  Mean-Variance Analysis in Portfolio Choice and Capital Markets , 1987 .

[40]  Alexander Shapiro,et al.  The Sample Average Approximation Method for Stochastic Discrete Optimization , 2002, SIAM J. Optim..

[41]  O. Nelles,et al.  An Introduction to Optimization , 1996, IEEE Antennas and Propagation Magazine.

[42]  Gábor Lugosi,et al.  Concentration Inequalities - A Nonasymptotic Theory of Independence , 2013, Concentration Inequalities.

[43]  László Györfi,et al.  A Probabilistic Theory of Pattern Recognition , 1996, Stochastic Modelling and Applied Probability.

[44]  A. Shapiro.  Stochastic Programming with Equilibrium Constraints , 2006 .

[45]  Alexander Shapiro,et al.  Lectures on Stochastic Programming: Modeling and Theory , 2009 .

[46]  Andreas Christmann,et al.  Support vector machines , 2008, Data Mining and Knowledge Discovery Handbook.

[47]  Johannes Jahn,et al.  Vector optimization - theory, applications, and extensions , 2004 .

[48]  J. Dupacová,et al.  ASYMPTOTIC BEHAVIOR OF STATISTICAL ESTIMATORS AND OF OPTIMAL SOLUTIONS OF STOCHASTIC OPTIMIZATION PROBLEMS , 1988 .

[49]  Michael Patriksson,et al.  Stochastic mathematical programs with equilibrium constraints , 1999, Oper. Res. Lett..

[50]  A. Shapiro.  Monte Carlo Sampling Methods , 2003 .

[51]  G. Pflug.  Stochastic Optimization and Statistical Inference , 2003 .

[52]  Melvyn J. Weeks,et al.  Simulation-based Inference in Econometrics: Simulation-based inference in econometrics: methods and applications , 2008 .

[53]  Peter Kall,et al.  Stochastic Programming , 1995 .

[54]  S. Boucheron,et al.  Theory of classification : a survey of some recent advances , 2005 .