A Model Reference Adaptive Search Method for Stochastic Global Optimization

We propose a randomized search method called Stochastic Model Reference Adaptive Search (SMRAS) for solving stochastic optimization problems in situations where the objective functions cannot be evaluated exactly, but can be estimated with some noise (or uncertainty), e.g., via simulation. The method is a generalization of the recently proposed Model Reference Adaptive Search (MRAS) method for deterministic global optimization, and is based on sampling from an underlying probability distribution \model" on the solution space, which is updated iteratively after evaluating the performance of the samples at each iteration. We show global convergence of SMRAS for both stochastic continuous and discrete (combinatorial) problems, and carry out numerical studies to illustrate the performance of the method.

[1]  J. Kiefer,et al.  Stochastic Estimation of the Maximum of a Regression Function , 1952 .

[2]  W. Hoeffding.  Probability Inequalities for sums of Bounded Random Variables , 1963 .

[3]  Peter W. Glynn,et al.  Likelilood ratio gradient estimation: an overview , 1987, WSC '87.

[4]  Sandro Ridella,et al.  Minimizing multimodal functions of continuous variables with the “simulated annealing” algorithmCorrigenda for this article is available here , 1987, TOMS.

[5]  Paul Glasserman,et al.  Gradient Estimation Via Perturbation Analysis , 1990 .

[6]  Xi-Ren Cao,et al.  Perturbation analysis of discrete event dynamic systems , 1991 .

[7]  Pierre L'Ecuyer,et al.  An overview of derivative estimation , 1991, 1991 Winter Simulation Conference Proceedings..

[8]  Leonard M. Adleman,et al.  Proof of proposition 3 , 1992 .

[9]  D. Yan,et al.  Stochastic discrete optimization , 1992 .

[10]  J. Spall.  Multivariate stochastic approximation using a simultaneous perturbation gradient approximation , 1992 .

[11]  Chuan Yi Tang,et al.  A 2.|E|-Bit Distributed Algorithm for the Directed Euler Trail Problem , 1993, Inf. Process. Lett..

[12]  Lalit M. Patnaik,et al.  Genetic algorithms: a survey , 1994, Computer.

[13]  Charles Leake,et al.  Discrete Event Systems: Sensitivity Analysis and Stochastic Optimization by the Score Function Method , 1994 .

[14]  Jason H. Goodfriend,et al.  Discrete Event Systems: Sensitivity Analysis and Stochastic Optimization by the Score Function Method , 1995 .

[15]  S. Andradóttir.  A method for discrete stochastic optimization , 1995 .

[16]  Gilbert Laporte,et al.  Annals of Operations Research , 1996 .

[17]  MICHAEL C. FU,et al.  Techniques for optimization via simulation: an experimental study on an (s,S) inventory system , 1997 .

[18]  Luca Maria Gambardella,et al.  Ant colony system: a cooperative learning approach to the traveling salesman problem , 1997, IEEE Trans. Evol. Comput..

[19]  M. R. Rao,et al.  Combinatorial Optimization , 1992, NATO ASI Series.

[20]  William J. Cook,et al.  Combinatorial optimization , 1997 .

[21]  George A. Vouros,et al.  Buffer allocation in unreliable production lines using a knowledge based system , 1998, Comput. Oper. Res..

[22]  R. Rubinstein.  The Cross-Entropy Method for Combinatorial and Continuous Optimization , 1999 .

[23]  S. Andradóttir,et al.  A Simulated Annealing Algorithm with Constant Temperature for Discrete Stochastic Optimization , 1999 .

[24]  Leyuan Shi,et al.  Nested Partitions Method for Stochastic Optimization , 2000 .

[25]  Pierre L'Ecuyer,et al.  Global Stochastic Optimization with Low-Dispersion Point Sets , 1998, Oper. Res..

[26]  Mauro Birattari,et al.  Model-based Search for Combinatorial Optimization , 2001 .

[27]  Roger W. Brockett,et al.  New Issues in the Mathematics of Control , 2000 .

[28]  Mahmoud H. Alrefaei,et al.  A modification of the stochastic ruler method for discrete stochastic optimization , 2001, Eur. J. Oper. Res..

[29]  Walter J. Gutjahr,et al.  A Converging ACO Algorithm for Stochastic Combinatorial Optimization , 2003, SAGA.

[30]  Peter Auer,et al.  Finite-time Analysis of the Multiarmed Bandit Problem , 2002, Machine Learning.

[31]  D. Wolpert.  Finding Bounded Rational Equilibria. Part 1; Iterative Focusing , 2004 .

[32]  Dirk P. Kroese,et al.  The Cross-Entropy Method: A Unified Approach to Combinatorial Optimization, Monte-Carlo Simulation and Machine Learning , 2004 .

[33]  Mauro Birattari,et al.  Model-Based Search for Combinatorial Optimization: A Critical Survey , 2004, Ann. Oper. Res..

[34]  Dirk P. Kroese,et al.  The Cross Entropy Method: A Unified Approach To Combinatorial Optimization, Monte-carlo Simulation (Information Science and Statistics) , 2004 .

[35]  Dirk P. Kroese,et al.  Application of the Cross-Entropy Method to the Buffer Allocation Problem in a Simulation-Based Environment , 2005, Ann. Oper. Res..

[36]  Jiaqiao Hu,et al.  A Model Reference Adaptive Search Algorithm for Global Optimization , 2005 .

[37]  Georg Ch. Pflug.  Sampling derivatives of probabilities , 2005, Computing.

[38]  Shie Mannor,et al.  A Tutorial on the Cross-Entropy Method , 2005, Ann. Oper. Res..

[39]  M. Fu.  Stochastic Gradient Estimation , 2005 .

[40]  Lih-Yuan Deng,et al.  The Cross-Entropy Method: A Unified Approach to Combinatorial Optimization, Monte-Carlo Simulation, and Machine Learning , 2006, Technometrics.

[41]  Barry L. Nelson,et al.  Discrete Optimization via Simulation Using COMPASS , 2006, Oper. Res..

[42]  H. Robbins.  A Stochastic Approximation Method , 1951 .

[43]  Michael C. Fu,et al.  A Model Reference Adaptive Search Method for Global Optimization , 2007, Oper. Res..

[44]  P. Glynn.  LIKELIHOOD RATIO GRADIENT ESTIMATION : AN OVERVIEW by , 2022 .