The cross-entropy method with patching for rare-event simulation of large Markov chains

There are various importance sampling schemes to estimate rare event probabilities in Markovian systems such as Markovian reliability models and Jackson networks. In this work, we present a general state-dependent importance sampling method which partitions the state space and applies the cross-entropy method to each partition. We investigate two versions of our algorithm and apply them to several examples of reliability and queueing models. In all these examples we compare our method with other importance sampling schemes. The performance of the importance sampling schemes is measured by the relative error of the estimator and by the efficiency of the algorithm. The results from experiments show considerable improvements both in running time of the algorithm and the variance of the estimator.

[1]  Tugrul Dayar,et al.  Comparison of Partitioning Techniques for Two-Level Iterative Solvers on Large, Sparse Markov Chains , 1999, SIAM J. Sci. Comput..

[2]  Christos Alexopoulos,et al.  Estimating reliability measures for highly-dependable Markov systems, using balanced likelihood ratios , 2001, IEEE Trans. Reliab..

[3]  Irina Ignatiouk-Robert.  Large deviations for processes with discontinuous statistics , 2004, math/0409392.

[4]  Perwez Shahabuddin,et al.  Importance sampling for the simulation of highly reliable Markovian systems , 1994 .

[5]  Thomas M. Cover,et al.  Elements of Information Theory: Cover/Elements of Information Theory, Second Edition , 2005 .

[6]  Philip Heidelberger,et al.  Fast simulation of rare events in queueing and reliability models , 1993, TOMC.

[7]  Paul Glasserman,et al.  Multilevel Splitting for Estimating Rare Event Probabilities , 1999, Oper. Res..

[8]  Perwez Shahabuddin,et al.  Fast Transient Simulation of Markovian Models of Highly Dependable Systems , 1994, Perform. Evaluation.

[9]  P. Shahabuddin,et al.  Chapter 11 Rare-Event Simulation Techniques: An Introduction and Recent Advances , 2006, Simulation.

[10]  Sigrún Andradóttir,et al.  On the Choice of Alternative Measures in Importance Sampling with Markov Chains , 1995, Oper. Res..

[11]  P. Dupuis,et al.  Dynamic importance sampling for queueing networks , 2007, 0710.4389.

[12]  Sandeep Juneja,et al.  Splitting-based importance-sampling algorithm for fast simulation of Markov reliability models with general repair-policies , 2001, IEEE Trans. Reliab..

[13]  S. Juneja,et al.  Rare-event Simulation Techniques : An Introduction and Recent Advances , 2006 .

[14]  Reuven Y. Rubinstein,et al.  Steady State Rare Events Simulation in Queueing Models and its Complexity Properties , 1994 .

[15]  Peter W. Glynn,et al.  Asymptotic robustness of estimators in rare-event simulation , 2007, TOMC.

[16]  W. Stewart.  Numerical Methods for Computing Stationary Distributions of Finite Irreducible Markov Chains , 2000 .

[17]  Ulrich Herzog,et al.  Formal Methods for Performance Evaluation , 2002, European Educational Forum: School on Formal Methods and Performance Analysis.

[18]  Donald L. Iglehart,et al.  Importance sampling for stochastic simulations , 1989 .

[19]  P. Ney,et al.  Monte Carlo simulation and large deviations theory for uniformly recurrent Markov chains , 1990, Journal of Applied Probability.

[20]  Reuven Y. Rubinstein,et al.  Simulation and the Monte Carlo method , 1981, Wiley series in probability and mathematical statistics.

[21]  George L. Nemhauser,et al.  Handbooks in operations research and management science , 1989 .

[22]  Adam Shwartz,et al.  Large Deviations For Performance Analysis , 2019 .

[23]  Pieter-Tjerk de Boer.  Analysis of state-independent importance-sampling measures for the two-node tandem queue , 2006, TOMC.

[24]  Marnix J. J. Garvels,et al.  The splitting method in rare event simulation , 2000 .

[25]  Pieter-Tjerk de Boer,et al.  Adaptive state- dependent importance sampling simulation of markovian queueing networks , 2002, Eur. Trans. Telecommun..

[26]  Gerardo Rubino,et al.  Introduction to Rare Event Simulation , 2009, Rare Event Simulation using Monte Carlo Methods.

[27]  Peter J. Smith.  Underestimation of Rare Event Probabilities in Importance Sampling Simulations , 2001, Simul..

[28]  Dirk P. Kroese,et al.  Simulation and the Monte Carlo method , 1981, Wiley series in probability and mathematical statistics.

[29]  Dennis D. Cox,et al.  Adaptive importance sampling on discrete Markov chains , 1999 .

[30]  Sandeep Juneja,et al.  Efficient simulation of buffer overflow probabilities in jackson networks with feedback , 2005, TOMC.

[31]  Michael Devetsikiotis,et al.  An algorithmic approach to the optimization of importance sampling parameters in digital communication system simulation , 1993, IEEE Trans. Commun..

[32]  Victor F. Nicola,et al.  Efficient importance sampling heuristics for the simulation of population overflow in Jackson networks , 2007, Proceedings of the Winter Simulation Conference, 2005..

[33]  P. Dupuis,et al.  Dynamic importance sampling for uniformly recurrent Markov chains , 2005, math/0503454.

[34]  Dirk P. Kroese,et al.  Simulation and the Monte Carlo Method (Wiley Series in Probability and Statistics) , 1981 .

[35]  R. S. Randhawa,et al.  Combining importance sampling and temporal difference control variates to simulate Markov Chains , 2004, TOMC.

[36]  Jack P. C. Kleijnen,et al.  Variance Reduction Techniques in Monte Carlo Methods , 2010 .

[37]  Ad Ridder,et al.  Importance Sampling Simulations of Markovian Reliability Systems Using Cross-Entropy , 2005, Ann. Oper. Res..

[38]  Dirk P. Kroese,et al.  Efficient simulation of a tandem Jackson network , 1999, TOMC.

[39]  Jean Walrand,et al.  A quick simulation method for excessive backlogs in networks of queues , 1989 .

[40]  Paul Glasserman,et al.  Analysis of an importance sampling estimator for tandem queues , 1995, TOMC.

[41]  Gerardo Rubino,et al.  Rare Event Simulation using Monte Carlo Methods , 2009 .

[42]  Michael R. Frater,et al.  Optimally efficient estimation of the statistics of rare events in queueing networks , 1991 .

[43]  Philip Heidelberger,et al.  A Unified Framework for Simulating Markovian Models of Highly Dependable Systems , 1992, IEEE Trans. Computers.

[44]  Dirk P. Kroese,et al.  A Fast Cross-Entropy Method for Estimating Buffer Overflows in Queueing Networks , 2004, Manag. Sci..

[45]  Sandeep Juneja,et al.  Fast Simulation of Markov Chains with Small Transition Probabilities , 2001, Manag. Sci..

[46]  Peter W. Glynn,et al.  A Markov chain perspective on adaptive Monte Carlo algorithms , 2001, Proceeding of the 2001 Winter Simulation Conference (Cat. No.01CH37304).

[47]  Pierre L'Ecuyer,et al.  EFFECTIVE APPROXIMATION OF ZERO-VARIANCE SIMULATION IN A RELIABILITY SETTING , 2007 .

[48]  Vivek S. Borkar,et al.  Adaptive Importance Sampling Technique for Markov Chains Using Stochastic Approximation , 2006, Oper. Res..

[49]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[50]  Lih-Yuan Deng,et al.  The Cross-Entropy Method: A Unified Approach to Combinatorial Optimization, Monte-Carlo Simulation, and Machine Learning , 2006, Technometrics.