Variable-Number Sample-Path Optimization

The sample-path method is one of the most important tools in simulation-based optimization. The basic idea of the method is to approximate the expected simulation output by the average of sample observations with a common random number sequence. In this paper, we describe a new variant of Powell’s unconstrained optimization by quadratic approximation (UOBYQA) method, which integrates a Bayesian variable-number sample-path (VNSP) scheme to choose appropriate number of samples at each iteration. The statistically accurate scheme determines the number of simulation runs, and guarantees the global convergence of the algorithm. The VNSP scheme saves a significant amount of simulation operations compared to general purpose ‘fixed-number’ sample-path methods. We present numerical results based on the new algorithm.

[1]  M. Degroot.  Optimal Statistical Decisions , 1970 .

[2]  A. Dawid.  Some matrix-variate distribution theory: Notational considerations and a Bayesian application , 1981 .

[3]  Jorge J. Moré,et al.  Recent Developments in Algorithms and Software for Trust Region Methods , 1982, ISMP.

[4]  Averill M. Law,et al.  Simulation Modeling and Analysis , 1982 .

[5]  Jorge J. Moré,et al.  Computing a Trust Region Step , 1983 .

[6]  R. Durrett.  Probability: Theory and Examples , 1993 .

[7]  Alexander Shapiro,et al.  Asymptotic Behavior of Optimal Solutions in Stochastic Programming , 1993, Math. Oper. Res..

[8]  Gül Gürkan,et al.  Sample-path optimization in simulation , 1994, Proceedings of Winter Simulation Conference.

[9]  Michael C. Fu,et al.  Optimization via simulation: A review , 1994, Ann. Oper. Res..

[10]  Peter Kall,et al.  Stochastic Programming , 1995 .

[11]  F. Giannessi,et al.  Nonlinear Optimization and Applications , 1996, Springer US.

[12]  P. Toint,et al.  An Algorithm using Quadratic Interpolation for Unconstrained Derivative Free Optimization , 1996 .

[13]  Stephen M. Robinson,et al.  Sample-path optimization of convex stochastic performance functions , 1996, Math. Program..

[14]  S. M. Robinson,et al.  Sample-path solution of stochastic variational inequalities, with applications to option pricing , 1996, Proceedings Winter Simulation Conference.

[15]  Stephen M. Robinson,et al.  Analysis of Sample-Path Optimization , 1996, Math. Oper. Res..

[16]  Katya Scheinberg,et al.  On the convergence of derivative-free methods for unconstrained optimization , 1997 .

[17]  Alexander Shapiro,et al.  A simulation-based approach to two-stage stochastic programming with recourse , 1998, Math. Program..

[18]  Stephen J. Wright,et al.  Numerical Optimization , 2018, Fundamental Statistical Inference.

[19]  Chun-Hung Chen,et al.  An asymptotic allocation for simultaneous simulation experiments , 1999, WSC'99. 1999 Winter Simulation Conference Proceedings. 'Simulation - A Bridge to the Future' (Cat. No.99CH37038).

[20]  Gül Gürkan,et al.  Solving stochastic optimization problems with stochastic constraints: an application in network design , 1999, WSC '99.

[21]  Russell R. Barton,et al.  Proceedings of the 2000 winter simulation conference , 2000 .

[22]  A. Shapiro.  Statistical Inference of Stochastic Optimization Problems , 2000 .

[23]  Michael C. Ferris,et al.  A practical approach to sample-path simulation optimization , 2000, 2000 Winter Simulation Conference Proceedings (Cat. No.00CH37165).

[24]  S. Uryasev.  Probabilistic constrained optimization : methodology and applications , 2000 .

[25]  Stephen E. Chick,et al.  New Procedures to Select the Best Simulated System Using Common Random Numbers , 2001, Manag. Sci..

[26]  Stephen E. Chick,et al.  New Two-Stage and Sequential Procedures for Selecting the Best Simulated System , 2001, Oper. Res..

[27]  M. J. D. Powell,et al.  UOBYQA: unconstrained optimization by quadratic approximation , 2002, Math. Program..

[28]  Alexander Shapiro,et al.  The Sample Average Approximation Method for Stochastic Discrete Optimization , 2002, SIAM J. Optim..

[29]  Jorge Nocedal,et al.  Wedge trust region methods for derivative free optimization , 2002, Math. Program..

[30]  A. Shapiro.  Monte Carlo Sampling Methods , 2003 .

[31]  Abhijit Gosavi,et al.  Simulation-Based Optimization: Parametric Optimization Techniques and Reinforcement Learning , 2003 .

[32]  A. Shapiro.  Monte Carlo sampling approach to stochastic programming , 2003 .

[33]  Tito Homem-de-Mello,et al.  Variable-sample methods for stochastic optimization , 2003, TOMC.

[34]  A. Ruszczynski.  Stochastic Programming Models , 2003 .

[35]  Barry L. Nelson,et al.  Selecting the best system: theory and methods , 2003, Proceedings of the 2003 Winter Simulation Conference, 2003..

[36]  Abhijit Gosavi,et al.  Simulation-Based Optimization: Parametric Optimization Techniques and Reinforcement Learning , 2003 .

[37]  Michael C. Ferris,et al.  Adaptation of the Uobyqa Algorithm for Noisy Functions , 2006, Proceedings of the 2006 Winter Simulation Conference.

[38]  Tito Homem-de-Mello.  On Rates of Convergence for Stochastic Optimization Problems Under Non-I.I.D. Sampling , 2006 .

[39]  M. Powell.  The NEWUOA software for unconstrained optimization without derivatives , 2006 .

[40]  Michael C. Ferris,et al.  Extension of the direct optimization algorithm for noisy functions , 2007, 2007 Winter Simulation Conference.

[41]  Geng Deng,et al.  Design optimization of a robust sleeve antenna for hepatic microwave ablation , 2008, Physics in medicine and biology.