Dynamic importance sampling for uniformly recurrent Markov chains

Importance sampling is a variance reduction technique for efficient estimation of rare-event probabilities by Monte Carlo. In standard importance sampling schemes, the system is simulated using an a priori fixed change of measure suggested by a large deviation lower bound analysis. Recent work, however, has suggested that such schemes do not work well in many situations. In this paper we consider dynamic importance sampling in the setting of uniformly recurrent Markov chains. By ``dynamic'' we mean that in the course of a single simulation, the change of measure can depend on the outcome of the simulation up till that time. Based on a control-theoretic approach to large deviations, the existence of asymptotically optimal dynamic schemes is demonstrated in great generality. The implementation of the dynamic schemes is carried out with the help of a limiting Bellman equation. Numerical examples are presented to contrast the dynamic and standard schemes.

[1]  J. Doob.  Stochastic processes , 1953 .

[2]  Michel Loève,et al.  Probability Theory I , 1977 .

[3]  D. Siegmund.  Importance Sampling in the Monte Carlo Study of Sequential Tests , 1976 .

[4]  丸山 徹.  Convex Analysisの二,三の進展について , 1977 .

[5]  Marie Cottrell,et al.  Large deviations and rare events in the study of stochastic algorithms , 1983 .

[6]  S. Asmussen.  Conjugate processes and the silumation of ruin problems , 1985 .

[7]  P. Ney,et al.  Large deviations of uniformly recurrent Markov additive processes , 1985 .

[8]  T. L. Lim,et al.  SELECTED AREAS IN , 1985 .

[9]  Harold J. Kushner,et al.  Stochastic systems with small noise, analysis and simulation; a phase locked loop example , 1985 .

[10]  P. Ney,et al.  MARKOV ADDITIVE PROCESSES II. LARGE DEVIATIONS , 1987 .

[11]  P. Ney,et al.  Markov Additive Processes I. Eigenvalue Properties and Limit Theorems , 1987 .

[12]  S. Asmussen.  Risk theory in a Markovian environment , 1989 .

[13]  James A. Bucklew,et al.  Large Deviation Techniques in Decision, Simulation, and Estimation , 1990 .

[14]  J.S. Sadowsky,et al.  On large deviations theory and asymptotically efficient Monte Carlo estimation , 1990, IEEE Trans. Inf. Theory.

[15]  H. Kushner.  Weak Convergence Methods and Singularly Perturbed Stochastic Control and Filtering Problems , 1990 .

[16]  A. Acosta.  Large deviations for empirical measures of Markov chains , 1990 .

[17]  P. Ney,et al.  Monte Carlo simulation and large deviations theory for uniformly recurrent Markov chains , 1990, Journal of Applied Probability.

[18]  Jean C. Walrand,et al.  Review of 'Large Deviation Techniques in Decision, Simulation, and Estimation' (Bucklew, J.A.; 1990) , 1991, IEEE Trans. Inf. Theory.

[19]  J. Sadowsky.  Large deviations theory and efficient simulation of excessive backlogs in a GI/GI/m queue , 1991 .

[20]  T. Lehtonen,et al.  SIMULATING LEVEL-CROSSING PROBABILITIES BY IMPORTANCE SAMPLING , 1992 .

[21]  T. Lehtonen,et al.  On asymptotically efficient simulation of ruin probabilities in a Markovian environment , 1992 .

[22]  John S. Sadowsky,et al.  On the optimality and stability of exponential twisting in Monte Carlo estimation , 1993, IEEE Trans. Inf. Theory.

[23]  Kung Yao,et al.  On Importance Sampling in Digital Communications - Part I: Fundamentals , 1993, IEEE J. Sel. Areas Commun..

[24]  Perwez Shahabuddin,et al.  Importance sampling for the simulation of highly reliable Markovian systems , 1994 .

[25]  R. Rubinstein,et al.  Regenerative rare events simulation via likelihood ratios , 1994, Journal of Applied Probability.

[26]  P. Glynn.  Large Deviations for the Infinite Server Queue in Heavy Traffic , 1995 .

[27]  Philip Heidelberger,et al.  Fast simulation of rare events in queueing and reliability models , 1993, TOMC.

[28]  Paul Glasserman,et al.  Analysis of an importance sampling estimator for tandem queues , 1995, TOMC.

[29]  J. Sadowsky.  On Monte Carlo estimation of large deviations probabilities , 1996 .

[30]  J. Lynch,et al.  A weak convergence approach to the theory of large deviations , 1997 .

[31]  P. Glasserman,et al.  Counterexamples in importance sampling for large deviations probabilities , 1997 .

[32]  James A. Bucklew.  The Blind Simulation Problem and Regenerative Processes , 1998, IEEE Trans. Inf. Theory.

[33]  Jeffrey F. Collamore.  Importance Sampling Techniques for the Multidimensional Ruin Problem for General Markov Additive Sequences of Random Vectors , 2002 .

[34]  Paul Glasserman,et al.  Monte Carlo Methods in Financial Engineering , 2003 .

[35]  P. Dupuis,et al.  Importance Sampling, Large Deviations, and Differential Games , 2004 .

[36]  Dimitri P. Bertsekas,et al.  Stochastic optimal control : the discrete time case , 2007 .