Computing Densities for Markov Chains via Simulation

We introduce a new class of density estimators, termed look-ahead density estimators, for performance measures associated with a Markov chain. Look-ahead density estimators are given for both transient and steady-state quantities. Look-ahead density estimators converge faster (especially in multidimensional problems) and empirically give visually superior results relative to more standard estimators, such as kernel density estimators. Several numerical examples that demonstrate the potential applicability of look-ahead density estimation are given.

[1]  Donald L. Iglehart,et al.  Simulating Stable Stochastic Systems, VI: Quantile Estimation , 1976, JACM.

[2]  George A. F. Seber,et al.  Linear regression analysis , 1977 .

[3]  Ing Rj Ser.  Approximation Theorems of Mathematical Statistics , 1980 .

[4]  A. F. Seila,et al.  A Batching Approach to Quantile Estimation in Regenerative Simulations , 1982 .

[5]  Prakasa Rao.  Nonparametric functional estimation , 1983 .

[6]  Paul Bratley,et al.  A guide to simulation , 1983 .

[7]  Philip Heidelberger,et al.  Quantile Estimation in Dependent Sequences , 1984, Oper. Res..

[8]  S. Yakowitz.  Nonparametric Density Estimation, Prediction, and Regression for Markov Sequences , 1985 .

[9]  Paul Bratley,et al.  A guide to simulation (2nd ed.) , 1986 .

[10]  R. F. Kappenman.  Improved distribution quantile estimation , 1987 .

[11]  L. Devroye.  A Course in Density Estimation , 1987 .

[12]  Sik-Yum Lee,et al.  Two-step estimation of multivariate polychoric correlation , 1987 .

[13]  Upendra Dave,et al.  Applied Probability and Queues , 1987 .

[14]  L. Devroye,et al.  Nonparametric density estimation : the L[1] view , 1987 .

[15]  Donald L. Iglehart,et al.  Simulation methods for queues: An overview , 1988, Queueing Syst. Theory Appl..

[16]  A. W. Kemp,et al.  Applied Probability and Queues , 1989 .

[17]  S. Yakowitz.  Nonparametric density and regression estimation for Markov sequences without mixing assumptions , 1989 .

[18]  David W. Scott,et al.  Multivariate Density Estimation: Theory, Practice, and Visualization , 1992, Wiley Series in Probability and Statistics.

[19]  Richard L. Tweedie,et al.  Markov Chains and Stochastic Stability , 1993, Communications and Control Engineering Series.

[20]  Paul Glasserman,et al.  Filtered Monte Carlo , 1993, Math. Oper. Res..

[21]  P. Glynn,et al.  Likelihood ratio gradient estimation for stochastic recursions , 1995, Advances in Applied Probability.

[22]  James R. Wilson,et al.  Correlation-induction techniques for estimating quantiles in simulation experiments , 1995, WSC '95.

[23]  James R. Wilson,et al.  Integrated Variance Reduction Strategies for Simulation , 1996, Oper. Res..

[24]  Sylvia Richardson,et al.  Markov Chain Monte Carlo in Practice , 1997 .

[25]  Michael C. Fu,et al.  Conditional Monte Carlo , 1997 .

[26]  P. Glynn,et al.  A batch means methodology for estimation of a nonlinear function of a steady-state mean , 1997 .

[27]  Peter W. Glynn,et al.  Estimation of stationary densities for Markov chains , 1998, 1998 Winter Simulation Conference. Proceedings (Cat. No.98CH36274).

[28]  B. Nelson,et al.  Control Variates for Probability and Quantile Estimation , 1998 .

[29]  James R. Wilson,et al.  Correlation-Induction Techniques for Estimating Quantiles in Simulation Experiments , 1998, Oper. Res..

[30]  Peter W. Glynn,et al.  Multivariate Standardized Time Series for Steady-State Simulation Output Analysis , 2001, Oper. Res..

[31]  Peter W. Glynn,et al.  Regenerative steady-state simulation of discrete-event systems , 2001, TOMC.