Variance reduction through smoothing and control variates for Markov chain simulations

We consider the simulation of a discrete Markov chain that is so large that numerical solution of the steady-state balance equations cannot be done with available computers, We propose smoothing methods to obtain variance reduction when simulation is used to estimate a function of a subset of the steady-state probabilities. These methods attempt to make each transition provide information about the probabilities of interest. We give an algorithm that converges to the optimal smoothing operator, and some guidelines for picking the parameters of this algorithm. Analytical arguments are used to justify our procedures, and they are buttressed by the results of a numerical example,

[1]  Donald L. Iglehart,et al.  Importance sampling for stochastic simulations , 1989 .

[2]  Philip Heidelberger.  Variance Reduction Techniques for the Simulation of Markov Processes, I: Multiple Estimates , 1980, IBM J. Res. Dev..

[3]  J. Keilson,et al.  A central limit theorem for processes defined on a finite Markov chain , 1964, Mathematical Proceedings of the Cambridge Philosophical Society.

[4]  J. Kleijnen.  Communication: Reply to Fox and Schruben , 1978 .

[5]  J. Keilson,et al.  A process with chain dependent growth rate , 1970, Journal of Applied Probability.

[6]  John G. Kemeny,et al.  Finite Markov chains , 1960 .

[7]  R. Cuninghame-Green,et al.  Applied Linear Algebra , 1979 .

[8]  William Aiello,et al.  Sparse Matrix Computations on Parallel Processor Arrays , 1993, SIAM J. Sci. Comput..

[9]  Daniel P. Heyman.  A Performance Model of the Credit Manager Algorithm , 1992, Comput. Networks ISDN Syst..

[10]  Walter L. Smith,et al.  Regenerative stochastic processes , 1955, Proceedings of the Royal Society of London. Series A. Mathematical and Physical Sciences.

[11]  John G. Kemeny,et al.  Finite Markov Chains. , 1960 .

[12]  B. Noble.  Applied Linear Algebra , 1969 .

[13]  J. Sadowsky.  Large deviations theory and efficient simulation of excessive backlogs in a GI/GI/m queue , 1991 .

[14]  Daniel P. Heyman,et al.  Stochastic models in operations research , 1982 .

[15]  Sigrún Andradóttir,et al.  Smoothing methods for variance reduction in simulation of Markov chains , 1992, WSC '92.

[16]  P. Glynn,et al.  Varaince reduction in mean time to failure simulations , 1988, 1988 Winter Simulation Conference Proceedings.

[17]  Robert J. Vanderbei,et al.  Quicker Convergence for Iterative Numerical Solutions to Stochastic Problems: Probabilistic Interpretations, Ordering Heuristics, and Parallel Processing , 1990 .

[18]  Paul Bratley,et al.  A guide to simulation (2nd ed.) , 1986 .