Explorer Greatest Fixed Points of Probabilistic Min / Max Polynomial Equations , and Reachability for Branching Markov Decision Processes ?

We give polynomial time algorithms for quantitative (and qualitative) reachability analysis for Branching Markov Decision Processes (BMDPs). Speci cally, given a BMDP, and given an initial population, where the objective of the controller is to maximize (or minimize) the probability of eventually reaching a population that contains an object of a desired (or undesired) type, we give algorithms for approximating the supremum (in mum) reachability probability, within desired precision > 0, in time polynomial in the encoding size of the BMDP and in log(1/ ). We furthermore give P-time algorithms for computing -optimal strategies for both maximization and minimization of reachability probabilities. We also give P-time algorithms for all associated qualitative analysis problems, namely: deciding whether the optimal (supremum or in mum) reachability probabilities are 0 or 1. Prior to this paper, approximation of optimal reachability probabilities for BMDPs was not even known to be decidable. Our algorithms exploit the following basic fact: we show that for any BMDP, its maximum (minimum) non-reachability probabilities are given by the greatest xed point (GFP) solution g∗ ∈ [0, 1] of a corresponding monotone max (min) Probabilistic Polynomial System of equations (max/min-PPS), x = P (x), which are the Bellman optimality equations for a BMDP with non-reachability objectives. We show how to compute the GFP of max/min PPSs to desired precision in P-time.

[1]  Kousha Etessami,et al.  Recursive Markov Decision Processes and Recursive Stochastic Games , 2005, ICALP.

[2]  Stefan Kiefer,et al.  Analysis of Probabilistic Basic Parallel Processes , 2014, FoSSaCS.

[3]  Krishnendu Chatterjee,et al.  Evolutionary dynamics of cancer in response to targeted combination therapy , 2013, eLife.

[4]  Krishnendu Chatterjee,et al.  TTP: Tool for Tumor Progression , 2013, CAV.

[5]  Stefan Kiefer,et al.  Model Checking Stochastic Branching Processes , 2012, MFCS.

[6]  Kousha Etessami,et al.  Polynomial Time Algorithms for Branching Markov Decision Processes and Probabilistic Min(Max) Polynomial Bellman Equations , 2012, ICALP.

[7]  Kousha Etessami,et al.  Polynomial Time Algorithms for Multi-Type Branching Processes and Stochastic Context-Free Grammars , 2012, ArXiv.

[8]  Tomás Brázdil,et al.  Qualitative reachability in stochastic BPA games , 2011, Inf. Comput..

[9]  Kousha Etessami,et al.  Recursive Markov chains, stochastic grammars, and monotone systems of nonlinear equations , 2005, JACM.

[10]  Kousha Etessami,et al.  Recursive Stochastic Games with Positive Rewards , 2008, ICALP.

[11]  Helmut Seidl,et al.  Approximative Methods for Monotone Systems of Min-Max-Polynomial Equations , 2008, ICALP.

[12]  Tomás Brázdil,et al.  Reachability in recursive Markov decision processes , 2008, Inf. Comput..

[13]  Javier Esparza,et al.  Model checking probabilistic pushdown automata , 2004, Proceedings of the 19th Annual IEEE Symposium on Logic in Computer Science, 2004..

[14]  Peter Whittle,et al.  Growth Optimality for Branching Markov Decision Chains , 1982, Math. Oper. Res..

[15]  S. Pliska.  Optimization of Multitype Branching Processes , 1976 .