Alternating Weak Automata from Universal Trees

This paper studies parametric Markov decision processes (pMDPs), an extension to Markov decision processes (MDPs) where transitions probabilities are described by polynomials over a finite set of parameters. Fixing values for all parameters yields MDPs. In particular, this paper studies the complexity of finding values for these parameters such that the induced MDP satisfies some reachability constraints. We discuss different variants depending on the comparison operator in the constraints and the domain of the parameter values. We improve all known lower bounds for this problem, and notably provide ETR-completeness results for distinct variants of this problem. Furthermore, we provide insights in the functions describing the induced reachability probabilities, and how pMDPs generalise concurrent stochastic reachability games.

[1]  Andrea Maggiolo-Schettini,et al.  Parametric probabilistic transition systems for system design and analysis , 2007, Formal Aspects of Computing.

[2]  Peter Norvig,et al.  Artificial Intelligence: A Modern Approach , 1995 .

[3]  Luca Bortolussi,et al.  Bayesian Statistical Parameter Synthesis for Linear Temporal Properties of Stochastic Models , 2018, TACAS.

[4]  Carlo Ghezzi,et al.  Supporting Self-Adaptation via Quantitative Verification and Sensitivity Analysis at Run Time , 2016, IEEE Transactions on Software Engineering.

[5]  Nils Jansen,et al.  Minimal counterexamples for linear-time probabilistic verification , 2014, Theor. Comput. Sci..

[6]  Nils Jansen,et al.  Accelerating Parametric Probabilistic Verification , 2014, QEST.

[7]  Christel Baier,et al.  Principles of model checking , 2008 .

[8]  Christel Baier,et al.  Parametric Markov Chains: PCTL Complexity and Fraction-free Gaussian Elimination , 2017, GandALF.

[9]  Di Wu,et al.  Reachability analysis of uncertain systems using bounded-parameter Markov decision processes , 2008, Artif. Intell..

[10]  Jeremy Sproston.  Qualitative Reachability for Open Interval Markov Chains , 2018, RP.

[11]  C. J. Himmelberg,et al.  Existence of p-equilibrium and optimal stationary strategies in stochastic games , 1976 .

[12]  Daniel Kuhn,et al.  Robust Markov Decision Processes , 2013, Math. Oper. Res..

[13]  Christel Baier,et al.  Parametric Markov chains: PCTL complexity and fraction-free Gaussian elimination , 2020, Inf. Comput..

[14]  Lijun Zhang,et al.  Synthesis for PCTL in Parametric Markov Decision Processes , 2011, NASA Formal Methods.

[15]  Lijun Zhang,et al.  Model Repair for Markov Decision Processes , 2013, 2013 International Symposium on Theoretical Aspects of Software Engineering.

[16]  Alberto L. Sangiovanni-Vincentelli,et al.  Polynomial-Time Verification of PCTL Properties of MDPs with Convex Uncertainties , 2013, CAV.

[17]  Kim Guldstrand Larsen,et al.  Specification and refinement of probabilistic processes , 1991, [1991] Proceedings Sixth Annual IEEE Symposium on Logic in Computer Science.

[18]  D. Vere-Jones.  Markov Chains , 1972, Nature.

[19]  Sven Schewe,et al.  Accelerated Model Checking of Parametric Markov Chains , 2018, ATVA.

[20]  Lubos Brim,et al.  Precise parameter synthesis for stochastic biochemical systems , 2014, Acta Informatica.

[21]  Krishnendu Chatterjee,et al.  Robustness of Structurally Equivalent Concurrent Parity Games , 2011, FoSSaCS.

[22]  Lijun Zhang,et al.  Probabilistic reachability for parametric Markov models , 2010, International Journal on Software Tools for Technology Transfer.

[23]  C. R. Ramakrishnan,et al.  Model Repair for Probabilistic Systems , 2011, TACAS.

[24]  Neil Immerman,et al.  The Complexity of Decentralized Control of Markov Decision Processes , 2000, UAI.

[25]  Marcus Schaefer,et al.  Fixed Points, Nash Equilibria, and the Existential Theory of the Reals , 2017, Theory of Computing Systems.

[26]  Sebastian Junges,et al.  Counterexample-Driven Synthesis for Probabilistic Program Sketches , 2019, FM.

[27]  Sebastian Junges,et al.  A Storm is Coming: A Modern Probabilistic Model Checker , 2017, CAV.

[28]  Sebastian Junges,et al.  Parameter Synthesis for Markov Models , 2019, Formal Methods Syst. Des..

[29]  James Renegar,et al.  On the Computational Complexity and Geometry of the First-Order Theory of the Reals, Part I: Introduction. Preliminaries. The Geometry of Semi-Algebraic Sets. The Decision Problem for the Existential Theory of the Reals , 1992, J. Symb. Comput..

[30]  Kristoffer Arnsfelt Hansen,et al.  Strategy Complexity of Concurrent Safety Games , 2017, MFCS.

[31]  Krishnendu Chatterjee,et al.  A survey of stochastic ω-regular games , 2012, J. Comput. Syst. Sci..

[32]  Martin L. Puterman,et al.  Markov Decision Processes: Discrete Stochastic Dynamic Programming , 1994 .

[33]  Conrado Daws.  Symbolic and Parametric Model Checking of Discrete-Time Markov Chains , 2004, ICTAC.

[34]  Anne Condon,et al.  Computational models of games , 1989, ACM distinguished dissertations.

[35]  David S. Rosenblum,et al.  Perturbation Analysis in Verification of Discrete-Time Markov Chains , 2014, CONCUR.

[36]  Marcus Schaefer,et al.  Realizability of Graphs and Linkages , 2013 .

[37]  U. Rieder,et al.  Markov Decision Processes , 2010 .

[38]  T. Parthasarathy.  Discounted and positive stochastic games , 1971 .

[39]  Ventsislav Chonev,et al.  Reachability in Augmented Interval Markov Chains , 2017, RP.

[40]  Ana Sokolova,et al.  SEA-PARAM: Exploring Schedulers in Parametric MDPs , 2017, QAPL@ETAPS.

[41]  Eilon Solan.  Continuity of the Value of Competitive Markov Decision Processes , 2003 .

[42]  Sebastian Junges,et al.  PROPhESY: A PRObabilistic ParamEter SYnthesis Tool , 2015, CAV.

[43]  Sebastian Junges,et al.  Permissive Finite-State Controllers of POMDPs using Parameter Synthesis , 2017, ArXiv.

[44]  Sebastian Junges,et al.  Shepherding Hordes of Markov Chains , 2019, TACAS.

[45]  Roberto Segala,et al.  Comparative analysis of bisimulation relations on alternating and non-alternating probabilistic models , 2005, Second International Conference on the Quantitative Evaluation of Systems (QEST'05).

[46]  Shlomo Zilberstein,et al.  Formal models and algorithms for decentralized decision making under uncertainty , 2008, Autonomous Agents and Multi-Agent Systems.

[47]  Laurent El Ghaoui,et al.  Robust Control of Markov Decision Processes with Uncertain Transition Matrices , 2005, Oper. Res..

[48]  VlassisNikos,et al.  On the Computational Complexity of Stochastic Controller Optimization in POMDPs , 2012 .

[49]  Scott Sanner,et al.  Solutions to Factored MDPs with Imprecise Transition Probabilities 1 , 2011 .

[50]  Andrew Chi-Chih Yao,et al.  The complexity of nonuniform random number generation , 1976 .

[51]  Marta Z. Kwiatkowska,et al.  PRISM 4.0: Verification of Probabilistic Real-Time Systems , 2011, CAV.

[52]  Mahesh Viswanathan,et al.  Model-Checking Markov Chains in the Presence of Uncertainties , 2006, TACAS.

[53]  Robert Givan,et al.  Bounded-parameter Markov decision processes , 2000, Artif. Intell..

[54]  Krishnendu Chatterjee,et al.  A Symbolic SAT-Based Algorithm for Almost-Sure Reachability with Small Strategies in POMDPs , 2015, AAAI.

[55]  Thomas A. Henzinger,et al.  Concurrent reachability games , 1998, Proceedings 39th Annual Symposium on Foundations of Computer Science (Cat. No.98CB36280).

[56]  David Barber,et al.  On the Computational Complexity of Stochastic Controller Optimization in POMDPs , 2011, TOCT.

[57]  Kristoffer Arnsfelt Hansen,et al.  Winning Concurrent Reachability Games Requires Doubly-Exponential Patience , 2009, 2009 24th Annual IEEE Symposium on Logic In Computer Science.

[58]  Sebastian Junges,et al.  Parameter Synthesis for Markov Models: Faster Than Ever , 2016, ATVA.

[59]  John F. Canny,et al.  Some algebraic and geometric computations in PSPACE , 1988, STOC '88.

[60]  Olle Häggström.  Finite Markov Chains and Algorithmic Applications , 2002 .

[61]  Taolue Chen,et al.  On the complexity of model checking interval-valued discrete time Markov chains , 2013, Inf. Process. Lett..

[62]  Pedro R. D'Argenio,et al.  Distributed probabilistic input/output automata: Expressiveness, (un)decidability and algorithms , 2014, Theor. Comput. Sci..

[63]  Sebastian Junges,et al.  Synthesis in pMDPs: A Tale of 1001 Parameters , 2018, ATVA.

[64]  Marie-Françoise Roy,et al.  Existential Theory of the Reals , 2003 .

[65]  D. Handelman.  Representing polynomials by positive linear functions on compact convex polyhedra. , 1988 .