Exact estimation for Markov chain equilibrium expectations

We introduce a new class of Monte Carlo methods, which we call exact estimation algorithms. Such algorithms provide unbiased estimators for equilibrium expectations associated with real- valued functionals defined on a Markov chain. We provide easily implemented algorithms for the class of positive Harris recurrent Markov chains, and for chains that are contracting on average. We further argue that exact estimation in the Markov chain setting provides a significant theoretical relaxation relative to exact simulation methods.

[1]  Kai Lai Chung,et al.  A Course in Probability Theory , 1949 .

[2]  K. Athreya,et al.  A New Approach to the Limit Theory of Recurrent Markov Chains , 1978 .

[3]  T. Rychlik.  Unbiased nonparametric estimation of the derivative of the mean , 1990 .

[4]  Ward Whitt,et al.  The Asymptotic Efficiency of Simulation Estimators , 1992, Oper. Res..

[5]  T. Lindvall.  Lectures on the Coupling Method , 1992 .

[6]  Peter W. Glynn,et al.  Stationarity detection in the initial transient problem , 1992, TOMC.

[7]  A. A. Borovkov,et al.  STOCHASTICALLY RECURSIVE SEQUENCES AND THEIR GENERALIZATIONS , 1992 .

[8]  Richard L. Tweedie,et al.  Markov Chains and Stochastic Stability , 1993, Communications and Control Engineering Series.

[9]  T. Rychlik.  A class of unbiased kernel estimates of a probability density function , 1995 .

[10]  David Bruce Wilson,et al.  Exact sampling with coupled Markov chains and applications to statistical mechanics , 1996, Random Struct. Algorithms.

[11]  J. Propp,et al.  Exact sampling with coupled Markov chains and applications to statistical mechanics , 1996 .

[12]  Persi Diaconis,et al.  Iterated Random Functions , 1999, SIAM Rev..

[13]  H. Thorisson.  Coupling, stationarity, and regeneration , 2000 .

[14]  P. Glynn,et al.  Simulating the maximum of a random walk , 2000 .

[15]  R. Tweedie,et al.  Perfect sampling of ergodic Harris chains , 2001 .

[16]  W. Kendall.  Geometric ergodicity and perfect simulation , 2004, math/0410012.

[17]  Peter W. Glynn,et al.  Stochastic Simulation: Algorithms and Analysis , 2007 .

[18]  Stephen B. Connor,et al.  Correction. Perfect simulation for a class of positive recurrent Markov chains , 2007, 0711.0804.

[19]  Michael B. Giles,et al.  Multilevel Monte Carlo Path Simulation , 2008, Oper. Res..

[20]  Don McLeish,et al.  A general method for debiasing a Monte Carlo estimator , 2010, Monte Carlo Methods Appl..

[21]  Peter W. Glynn,et al.  A new approach to unbiased estimation for SDE's , 2012, Proceedings Title: Proceedings of the 2012 Winter Simulation Conference (WSC).

[22]  Peter W. Glynn,et al.  Unbiased Estimation with Square Root Convergence for SDE Models , 2015, Oper. Res..