Limiting Behavior of Markov Chains with Eager Attractors

We consider discrete infinite-state Markov chains which contain an eager finite attractor. A finite attractor is a finite subset of states that is eventually reached with probability 1 from every other state, and the eagerness condition requires that the probability of avoiding the attractor in n or more steps after leaving it is exponentially bounded in n. Examples of such Markov chains are those induced by probabilistic lossy channel systems and similar systems. We show that the expected residence time (a generalization of the steady state distribution) exists for Markov chains with eager attractors and that it can be effectively approximated to arbitrary precision. Furthermore, arbitrarily close approximations of the limiting average expected reward, with respect to state-based bounded reward functions, are also computable

[1]  Moshe Y. Vardi.  Automatic verification of probabilistic concurrent finite state programs , 1985, 26th Annual Symposium on Foundations of Computer Science (sfcs 1985).

[2]  Christel Baier,et al.  Establishing Qualitative Properties for Probabilistic Lossy Channel Systems: An Algorithmic Approach , 1999, ARTS.

[3]  Tomás Brázdil,et al.  Computing the Expected Accumulated Reward and Gain for a Subclass of Infinite Markov Chains , 2005, FSTTCS.

[4]  J. Ben Atkinson,et al.  Modeling and Analysis of Stochastic Systems , 1996 .

[5]  Luca de Alfaro,et al.  Symbolic Model Checking of Probabilistic Processes Using MTBDDs and the Kronecker Representation , 2000, TACAS.

[6]  Parosh Aziz Abdulla,et al.  Verification of Probabilistic Systems with Faulty Communication , 2003, FoSSaCS.

[7]  Alain Finkel,et al.  Unreliable Channels are Easier to Verify Than Perfect Channels , 1996, Inf. Comput..

[8]  Joost-Pieter Katoen,et al.  Formal Methods for Real-Time and Probabilistic Systems , 1999, Lecture Notes in Computer Science.

[9]  Stephan Merz,et al.  Model Checking , 2000 .

[10]  Kousha Etessami,et al.  Recursive Markov Chains, Stochastic Grammars, and Monotone Systems of Nonlinear Equations , 2005, STACS.

[11]  Marta Z. Kwiatkowska,et al.  Probabilistic model checking in practice: case studies with PRISM , 2005, PERV.

[12]  Javier Esparza,et al.  Quantitative analysis of probabilistic pushdown automata: expectations and variances , 2005, 20th Annual IEEE Symposium on Logic in Computer Science (LICS' 05).

[13]  Parosh Aziz Abdulla,et al.  Verifying infinite Markov chains with a finite attractor or the global coarseness property , 2005, 20th Annual IEEE Symposium on Logic in Computer Science (LICS' 05).

[14]  Christel Baier,et al.  A note on the attractor-property of infinite-state Markov chains , 2006, Inf. Process. Lett..

[15]  Mihalis Yannakakis,et al.  The complexity of probabilistic verification , 1995, JACM.

[16]  Robert K. Brayton,et al.  Model-checking continuous-time Markov chains , 2000, TOCL.

[17]  Kousha Etessami,et al.  Verifying Probabilistic Procedural Programs , 2004, FSTTCS.

[18]  S. Purushothaman Iyer,et al.  Probabilistic Lossy Channel Systems , 1997, TAPSOFT.

[19]  Javier Esparza,et al.  Model checking probabilistic pushdown automata , 2004, Proceedings of the 19th Annual IEEE Symposium on Logic in Computer Science, 2004..

[20]  Christel Baier,et al.  Automated Performance and Dependability Evaluation Using Model Checking , 2002, Performance.

[21]  Alexander Moshe Rabinovich,et al.  Quantitative Analysis of Probabilistic Lossy Channel Systems , 2003, ICALP.

[22]  Parosh Aziz Abdulla,et al.  Eager Markov Chains , 2006, ATVA.

[23]  Nathalie Bertrand,et al.  Model Checking Lossy Channels Systems Is Probably Decidable , 2003, FoSSaCS.

[24]  Parosh Aziz Abdulla,et al.  Verifying Programs with Unreliable Channels , 1996, Inf. Comput..

[25]  Parosh Aziz Abdulla,et al.  Reasoning about Probabilistic Lossy Channel Systems , 2000, CONCUR.

[26]  Kousha Etessami,et al.  Algorithmic Verification of Recursive Probabilistic State Machines , 2005, TACAS.