Censoring Markov Chains and Stochastic Bounds

We show how to combine censoring technique for Markov chain and strong stochastic comparison to obtain bounds on rewards and the first passage time. We present the main ideas of the method, the algorithms and their proofs. We obtain a substantial reduction of the state space due to the censoring technique. We also present some numerical results to illustrate the effectiveness of the method.

[1]  A. Müller,et al.  Comparison Methods for Stochastic Models and Risks , 2002 .

[2]  Pierre Semal,et al.  Bounds for the Positive Eigenvectors of Nonnegative Matrices and for their Approximations by Decomposition , 1984, JACM.

[3]  Carl D. Meyer,et al.  Matrix Analysis and Applied Linear Algebra , 2000 .

[4]  Jean-Michel Fourneau,et al.  Transforming stochastic matrices for stochastic comparison with the st-order , 2003, RAIRO Oper. Res..

[5]  Terry Williams,et al.  Probability and Statistics with Reliability, Queueing and Computer Science Applications , 1983 .

[6]  Serge Haddad,et al.  Sub-stochastic matrix analysis for bounds computation - Theoretical results , 2007, Eur. J. Oper. Res..

[7]  Carl D. Meyer,et al.  Stochastic Complementation, Uncoupling Markov Chains, and the Theory of Nearly Reducible Systems , 1989, SIAM Rev..

[8]  J. Fourneau,et al.  Increasing Convex Monotone Markov Chains: Theory, Algorithm and Applications , 2005 .

[9]  J. Fourneau,et al.  Algorithms for an irreducible and lumpable strong stochastic bound , 2004 .

[10]  Jean-Michel Fourneau,et al.  An Algorithmic Approach to Stochastic Bounds , 2002, Performance.

[11]  Laurent Truffet.  Reduction techniques for discrete-time Markov chains on totally ordered state space using stochastic comparisons , 2000, Journal of Applied Probability.

[12]  Laurent Truffet.  Near Complete Decomposability: Bounding the Error by a Stochastic Comparison Method , 1997, Advances in Applied Probability.

[13]  Ana Busic,et al.  Bounds for Point and Steady-State Availability: An Algorithmic Approach Based on Lumpability and Stochastic Ordering , 2005, EPEW/WS-FM.

[14]  Danielle Liu,et al.  The censored Markov chain and the best augmentation , 1996, Journal of Applied Probability.

[15]  Nihal Pekergin,et al.  Componentwise bounds for nearly completely decomposable Markov chains using stochastic comparison and reordering , 2005, Eur. J. Oper. Res..

[16]  Conditional steady-state bounds for a subset of states in Markov chains , 2006, SMCtools '06.

[17]  Moshe Shaked,et al.  Stochastic orders and their applications , 1994 .

[18]  Jean-Michel Fourneau,et al.  An open tool to compute stochastic bounds on steady-state distributions and rewards , 2003, 11th IEEE/ACM International Symposium on Modeling, Analysis and Simulation of Computer Telecommunications Systems, 2003. MASCOTS 2003..

[19]  Kishor S. Trivedi.  Probability and Statistics with Reliability, Queuing, and Computer Science Applications , 1984 .