Stochastic Optimization by Simulation: Some Experiments with a Simple Steady-State Queue

Abstract : New approaches like perturbation analysis and the likelihood ratio method have been proposed recently to estimate the gradient of a performance measure with respect to some continuous parameters in a dynamic stochastic system. In this paper, we experiment the use of these estimators in stochastic approximation algorithms, to perform so-called single-run optimizations. We also compare them to finite difference estimators, with and without common random numbers. The experiments are done on a simple M/M/1 queue. The performance measure involves the average system time per customer, and the optimal solution is easy to compute analytically, which facilitates the evaluation of the algorithms. We also demonstrate some properties of the algorithms. In particular, we show that using perturbation analysis, the single-run optimization converges to the optimum even with a fixed (and small) number of ends of service per iteration, while under the same conditions, the algorithm that uses the finite difference estimators converges to the wrong answer.