Stochastic optimization by simulation: numerical experiments with the M / M /1 queue in steady-state

This paper gives numerical illustrations of the behavior of stochastic approximation, combined with different derivative estimation techniques, to optimize a steady-state system. It is a companion paper to L'Ecuyer and Glynn 1993, which gives convergence proofs for most of the variants experimented here. The numerical experiments are made with a simple M/M/1 queue, which while simple, serves to illustrate the basic convergence properties and Possible pitfalls of the various techniques.

[1]  K. Chung.  On a Stochastic Approximation Method , 1954 .

[2]  H. Kesten.  Accelerated Stochastic Approximation , 1958 .

[3]  J. H. Venter.  An extension of the Robbins-Monro procedure , 1967 .

[4]  V. Fabian.  On Asymptotic Normality in Stochastic Approximation , 1968 .

[5]  Michael Paul Griscik.  A New Algorithm for Stochastic Approximation , 1970 .

[6]  P. Révész,et al.  A limit theorem for the Robbins-Monro approximation , 1973 .

[7]  J. Talavage,et al.  Optimization of stochastic simulation models , 1980 .

[8]  Linus Schrage,et al.  A guide to simulation , 1983 .

[9]  Paul Bratley,et al.  A guide to simulation , 1983 .

[10]  Peter W. Glynn,et al.  Proceedings of Ihe 1986 Winter Simulation , 2022 .

[11]  Paul Bratley,et al.  A guide to simulation (2nd ed.) , 1986 .

[12]  Peter W. Glynn,et al.  Likelilood ratio gradient estimation: an overview , 1987, WSC '87.

[13]  Xi-Ren Cao,et al.  Convergence properties of infinitesimal perturbation analysis , 1988 .

[14]  R. Suri,et al.  Perturbation analysis gives strongly consistent sensitivity estimates for the M/G/ 1 queue , 1988 .

[15]  L. Goldstein.  On the choice of step size in the Robbins-Monro procedure , 1988 .

[16]  Rajan Suri,et al.  Single Run Optimization of Discrete Event Simulations—An Empirical Study Using the M/M/l Queue , 1989 .

[17]  P. Glynn,et al.  Stochastic Optimization by Simulation: Some Experiments with a Simple Steady-State Queue , 1989 .

[18]  P. Glynn.  Optimization of stochastic systems via simulation , 1989, WSC '89.

[19]  Sigrún Andradóttir.  A new algorithm for stochastic optimization , 1990, 1990 Winter Simulation Conference Proceedings.

[20]  M. Fu.  Convergence of a stochastic approximation algorithm for the GI/G/1 queue using infinitesimal perturbation analysis , 1990 .

[21]  Georg Ch. Pflug.  On-Line Optimization of Simulated Markovian Processes , 1990, Math. Oper. Res..

[22]  P. L’Ecuyer,et al.  A Unified View of the IPA, SF, and LR Gradient Estimation Techniques , 1990 .

[23]  P. Varaiya.  Convergence of a Stochastic Approximation Algorithm for the GI / G / 1 Queue Using Infinitesimal Perturbation Analysis , 1990 .

[24]  Paul Glasserman,et al.  Gradient Estimation Via Perturbation Analysis , 1990 .

[25]  Pierre L'Ecuyer,et al.  An overview of derivative estimation , 1991, 1991 Winter Simulation Conference Proceedings..

[26]  P. Dupuis,et al.  On sampling controlled stochastic approximation , 1991 .

[27]  G. Yin.  Stochastic Approximation Via Averaging: The Polyak’s Approach Revisited , 1992 .

[28]  A. Gaivoronski.  Optimization of Stochastic Discrete Event Dynamic Systems: A Survey of Some Recent Results , 1992 .

[29]  H. Kushner,et al.  Stochastic approximation with averaging of the iterates: Optimal asymptotic rate of convergence for , 1993 .

[30]  P. Glynn,et al.  Stochastic Optimization by Simulation: Convergence Proofs for the GI/G/1 Queue in Steady-State , 1994 .

[31]  E. Chong,et al.  Optimization of queues using an infinitesimal perturbation analysis-based stochastic algorithm with general update times , 1993 .

[32]  E. Chong,et al.  Stochastic optimization of regenerative systems using infinitesimal perturbation analysis , 1994, IEEE Trans. Autom. Control..