论文信息 - A consistent algorithm for derivative estimation of Markov chains

A consistent algorithm for derivative estimation of Markov chains

A consistent algorithm for derivative estimation of finite-state, discrete-time Markov chains is presented. The basic idea is to simulate original Markov chains with modified performance measures that can be estimated by extra simulation. The computational load of the extra simulation at each step is bounded. The algorithm attains the best possible rate of convergence as the simulation time goes to infinity. A connection between the algorithm and solutions to Poisson equations is also revealed.<<ETX>>

L. Dai

[1] J. Doob. Stochastic processes , 1953 .

[2] G. S. Fishman. Principles of Discrete Event Simulation , 1978 .

[3] S. Goldstein. Maximal coupling , 1979 .

[4] Wei-Bo Gong,et al. Smoothed Perturbation Analysis of Markovian Queueing Networks , 1988, 1988 American Control Conference.

[5] C. Cassandras,et al. On-line sensitivity analysis of Markov chains , 1989 .

[6] P. Glasserman. Discrete-time 'inversion' and derivative estimation for Markov chains , 1990 .

[7] Paul Glasserman,et al. Gradient Estimation Via Perturbation Analysis , 1990 .

[8] Xi-Ren Cao,et al. Perturbation analysis of discrete event dynamic systems , 1991 .

[9] B. Fox,et al. Estimating Derivatives Via Poisson's Equation , 1991, Probability in the Engineering and Informational Sciences.

[10] T. Lindvall. Lectures on the Coupling Method , 1992 .

[11] Christos G. Cassandras,et al. Discrete event systems : modeling and performance analysis , 1993 .

[12] Michael C. Fu,et al. Smoothed perturbation analysis derivative estimation for Markov chains , 1994, Oper. Res. Lett..

[13] Y. Ho,et al. Structural infinitesimal perturbation analysis (SIPA) for derivative estimation of discrete-event dynamic systems , 1995, IEEE Trans. Autom. Control..

[14] P. Brdmaud. Maximal coupling and rare perturbation sensitivity analysis , .