Decisive Markov Chains

We consider qualitative and quantitative verification problems for infinite- state Markov chains. We call a Markov chain decisive w.r.t. a given set of target states F if it almost certainly eventually reaches either F or a state from which F can no longer be reached. While all finite Markov chains are trivially decisive (for every set F), this also holds for many classes of infinite Markov chains. Infinite Markov chains which contain a finite attractor are decisive w.r.t. every set F. In particular, all Markov chains induced by probabilistic lossy channel systems (PLCS) con- tain a finite attractor and are thus decisive. Furthermore, all globally coarse Markov chains are decisive. The class of globally coarse Markov chains includes, e.g., those induced by probabilistic vector addition systems (PVASS) with upward-closed sets F, and all Markov chains induced by probabilistic noisy Turing machines (PNTM) (a generalization of the noisy Turing machines (NTM) of Asarin and Collins). We consider both safety and liveness problems for decisive Markov chains. Safety: What is the probability that a given set of states F is eventually reached. Liveness: What is the probability that a given set of states is reached infinitely often. There are three variants of these questions. (1) The qualitative problem, i.e., deciding if the probability is one (or zero); (2) the approximate quantitative problem, i.e., computing the probability up-to arbitrary precision; (3) the exact quantitative problem, i.e., computing probabilities exactly. 1. We express the qualitative problem in abstract terms for decisive Markov chains, and show an almost complete picture of its decidability for PLCS, PVASS and PNTM. 2. We also show that the path enumeration algorithm of Iyer and Narasimha terminates for decisive Markov chains and can thus be used to solve the approximate quantitative safety problem. A modified variant of this algorithm can be used to solve the approximate quantitative liveness problem. 3. Finally, we show that the exact probability of (repeatedly) reaching F cannot be effectively expressed (in a uniform way) in Tarski-algebra for either PLCS, PVASS or (P)NTM (unlike for probabilistic pushdown automata).

[1]  Christel Baier,et al.  Establishing Qualitative Properties for Probabilistic Lossy Channel Systems: An Algorithmic Approach , 1999, ARTS.

[2]  Parosh Aziz Abdulla,et al.  Reasoning about Probabilistic Lossy Channel Systems , 2000, CONCUR.

[3]  Jeffrey D. Ullman,et al.  Introduction to Automata Theory, Languages and Computation , 1979 .

[4]  Kousha Etessami,et al.  Algorithmic Verification of Recursive Probabilistic State Machines , 2005, TACAS.

[5]  Claude E. Shannon,et al.  Computability by Probabilistic Machines , 1970 .

[6]  J. Kemeny,et al.  Denumerable Markov chains , 1969 .

[7]  Petr Jancar,et al.  Decidability of a Temporal Logic Problem for Petri Nets , 1990, Theor. Comput. Sci..

[8]  Eugene Asarin,et al.  Noisy Turing Machines , 2005, ICALP.

[9]  Parosh Aziz Abdulla,et al.  Verifying Programs with Unreliable Channels , 1996, Inf. Comput..

[10]  Mihalis Yannakakis,et al.  The complexity of probabilistic verification , 1995, JACM.

[11]  Parosh Aziz Abdulla,et al.  Algorithmic Analysis of Programs with Well Quasi-ordered Domains , 2000, Inf. Comput..

[12]  Stephan Merz,et al.  Model Checking , 2000 .

[13]  Alexander Moshe Rabinovich,et al.  Quantitative Analysis of Probabilistic Lossy Channel Systems , 2003, ICALP.

[14]  Kousha Etessami,et al.  Recursive Markov Chains, Stochastic Grammars, and Monotone Systems of Nonlinear Equations , 2005, STACS.

[15]  Bruno Courcelle,et al.  On Constructing Obstruction Sets of Words , 1991, Bull. EATCS.

[16]  Kousha Etessami,et al.  Verifying Probabilistic Procedural Programs , 2004, FSTTCS.

[17]  Javier Esparza,et al.  Model checking probabilistic pushdown automata , 2004, Proceedings of the 19th Annual IEEE Symposium on Logic in Computer Science, 2004..

[18]  Graham Higman,et al.  Ordering by Divisibility in Abstract Algebras , 1952 .

[19]  A. Tarski.  A Decision Method for Elementary Algebra and Geometry , 2023 .

[20]  Nathalie Bertrand,et al.  Verifying Nondeterministic Channel Systems With Probabilistic Message Losses 3 , 2004 .

[21]  Aaron D. Wyner,et al.  Computability by Probabilistic Machines , 1993 .

[22]  Mihalis Yannakakis,et al.  Verifying temporal properties of finite-state probabilistic programs , 1988, [Proceedings 1988] 29th Annual Symposium on Foundations of Computer Science.

[23]  Kousha Etessami,et al.  Recursive Markov chains, stochastic grammars, and monotone systems of nonlinear equations , 2005, JACM.

[24]  Moshe Y. Vardi.  Automatic verification of probabilistic concurrent finite state programs , 1985, 26th Annual Symposium on Foundations of Computer Science (sfcs 1985).

[25]  Nathalie Bertrand,et al.  Model Checking Lossy Channels Systems Is Probably Decidable , 2003, FoSSaCS.

[26]  Peter Radford,et al.  Petri Net Theory and the Modeling of Systems , 1982 .

[27]  Saharon Shelah,et al.  Reasoning with Time and Chance , 1982, Inf. Control..

[28]  Michael Huth,et al.  Quantitative analysis and model checking , 1997, Proceedings of Twelfth Annual IEEE Symposium on Logic in Computer Science.

[29]  Daniel Brand,et al.  On Communicating Finite-State Machines , 1983, JACM.

[30]  L. Dickson.  Finiteness of the Odd Perfect and Primitive Abundant Numbers with n Distinct Prime Factors , 1913 .

[31]  Micha Sharir,et al.  Probabilistic temporal logics for finite and bounded models , 1984, STOC '84.

[32]  S. Purushothaman Iyer,et al.  Probabilistic Lossy Channel Systems , 1997, TAPSOFT.

[33]  Ernst W. Mayr.  An Algorithm for the General Petri Net Reachability Problem , 1984, SIAM J. Comput..

[34]  William Feller,et al.  An Introduction to Probability Theory and Its Applications , 1967 .

[35]  Parosh Aziz Abdulla,et al.  Verification of Probabilistic Systems with Faulty Communication , 2003, FoSSaCS.

[36]  Alain Finkel,et al.  Unreliable Channels are Easier to Verify Than Perfect Channels , 1996, Inf. Comput..

[37]  Joost-Pieter Katoen,et al.  Formal Methods for Real-Time and Probabilistic Systems , 1999, Lecture Notes in Computer Science.

[38]  Javier Esparza,et al.  Quantitative analysis of probabilistic pushdown automata: expectations and variances , 2005, 20th Annual IEEE Symposium on Logic in Computer Science (LICS' 05).

[39]  Parosh Aziz Abdulla,et al.  Verifying infinite Markov chains with a finite attractor or the global coarseness property , 2005, 20th Annual IEEE Symposium on Logic in Computer Science (LICS' 05).

[40]  Grégoire Sutre,et al.  An Optimal Automata Approach to LTL Model Checking of Probabilistic Systems , 2003, LPAR.