Means and variances of time averages in Markovian environments

Abstract Means and variances of time averages are important for cost minimization, hypothesis testing, estimation, initial bias problems in simulation and run length determination. This paper demonstrates how these measures can be calculated efficiently in transient Markovian systems. The method proposed here can be used to analyse systems of significant size, that is, systems with 1000 states or more. Numerical results are also given.

[1]  William Feller,et al.  An Introduction to Probability Theory and Its Applications , 1967 .

[2]  J. Keilson,et al.  A process with chain dependent growth rate , 1970, Journal of Applied Probability.

[3]  Winfried K. Grassmann.  Markov modelling , 1983, WSC '83.

[4]  W. Grassmann.  Transient solutions in Markovian queues : An algorithm for finding them and determining their waiting-time distributions , 1977 .

[5]  James R. Wilson,et al.  A survey of research on the simulation startup problem , 1978 .

[6]  Winfried K. Grassmann,et al.  Initial bias and estimation error in discrete event simulation , 1982, WSC.

[7]  Micha Yadin,et al.  Numerical Computation of Sojourn-Time Distributions in Queuing Networks , 1984, JACM.

[8]  Winfried K. Grassmann.  Transient solutions in markovian queueing systems , 1977, Comput. Oper. Res..

[9]  Micha Yadin,et al.  Randomization Procedures in the Computation of Cumulative-Time Distributions over Discrete State Markov Processes , 1984, Oper. Res..

[10]  A. Alan B. Pritsker,et al.  Formulas for the variance of the sample mean in finite state Markov processes , 1977, WSC '77.

[11]  Winfried K. Grassmann.  Technical Note - The Optimal Estimation of the Expected Number in a M/D/∞ Queueing System , 1981, Oper. Res..

[12]  P. Glynn.  Some asymptotic formulas for markov chains with applications to simulation , 1984 .

[13]  Shlomo Halfin.  Linear Estimators for a Class of Stationary Queueing Processes , 1982, Oper. Res..

[14]  Gordon F. Newell,et al.  A diffusion approximation for correlation in queues , 1980, Journal of Applied Probability.

[15]  Arthur F. Veinott,et al.  Analysis of Inventory Systems , 1963 .

[16]  A. Jensen,et al.  Markoff chains as an aid in the study of Markoff processes , 1953 .

[17]  Jürg Kohlas,et al.  Stochastic Methods of Operations Research , 1983 .

[18]  John F. Reynolds.  Some theorems on the transient covariance of Markov chains , 1972 .

[19]  J. Reynolds.  The covariance structure of queues and related processes – a survey of recent work , 1975, Advances in Applied Probability.

[20]  Donald Gross,et al.  The Randomization Technique as a Modeling Tool and Solution Procedure for Transient Markov Processes , 1984, Oper. Res..