Synthesis for PCTL in Parametric Markov Decision Processes

In parametric Markov decision processes (PMDPs), transition probabilities are not fixed, but are given as functions over a set of parameters. A PMDP denotes a family of concrete MDPs. This paper studies the synthesis problem for PCTL in PMDPs: Given a specification F in PCTL, we synthesise the parameter valuations under which F is true. First, we divide the possible parameter space into hyper-rectangles. We use existing decision procedures to check whether F holds on each of the Markov processes represented by the hyper-rectangle. As it is normally impossible to cover the whole parameter space by hyper-rectangles, we allow a limited area to remain undecided. We also consider an extension of PCTL with reachability rewards. To demonstrate the applicability of the approach, we apply our technique on a case study, using a preliminary implementation.

[1]  Rolf van Dawen,et al.  Finite state dynamic programming with the total reward criterion , 1986, Z. Oper. Research.

[2]  Jeffrey D. Ullman,et al.  Introduction to automata theory, languages, and computation, 2nd edition , 2001, SIGA.

[3]  Maurice Herlihy,et al.  Fast Randomized Consensus Using Shared Memory , 1990, J. Algorithms.

[4]  J. van der Wal,et al.  Stochastic dynamic programming : successive approximations and nearly optimal strategies for markov decision processes and markov games , 1981 .

[5]  Christel Baier,et al.  Model checking performability properties , 2002, Proceedings International Conference on Dependable Systems and Networks.

[6]  C. R. Ramakrishnan,et al.  Model Repair for Probabilistic Systems , 2011, TACAS.

[7]  Jeffrey D. Ullman,et al.  Introduction to Automata Theory, Languages and Computation , 1979 .

[8]  Frits W. Vaandrager,et al.  Cost-optimization of the IPv4 zeroconf protocol , 2003, 2003 International Conference on Dependable Systems and Networks, 2003. Proceedings..

[9]  Stefan Ratschan,et al.  Efficient solving of quantified inequality constraints over the real numbers , 2002, TOCL.

[10]  William H. Sanders,et al.  Optimal state-space lumping in Markov chains , 2003, Inf. Process. Lett..

[11]  Stefan Ratschan.  Safety Verification of Non-linear Hybrid Systems Is Quasi-Semidecidable , 2010, TAMC.

[12]  Laurent Fribourg,et al.  An Inverse Method for Policy-Iteration Based Algorithms , 2009, INFINITY.

[13]  Andrea Bianco,et al.  Model Checking of Probabalistic and Nondeterministic Systems , 1995, FSTTCS.

[14]  Martin L. Puterman,et al.  Markov Decision Processes: Discrete Stochastic Dynamic Programming , 1994 .

[15]  Conrado Daws.  Symbolic and Parametric Model Checking of Discrete-Time Markov Chains , 2004, ICTAC.

[16]  Christel Baier,et al.  Weak Bisimulation for Fully Probabilistic Processes , 1997, FBT.

[17]  Zhiming Liu,et al.  Theoretical Aspects of Computing - ICTAC 2004, First International Colloquium, Guiyang, China, September 20-24, 2004, Revised Selected Papers , 2005, ICTAC.

[18]  Christel Baier,et al.  Principles of Model Checking (Representation and Mind Series) , 2008 .

[19]  David Blackwell,et al.  On the functional equation of dynamic programming , 1961 .

[20]  Lijun Zhang,et al.  PARAM: A Model Checker for Parametric Markov Models , 2010, CAV.

[21]  Ulrich Herzog,et al.  Formal Methods for Performance Evaluation , 2002, European Educational Forum: School on Formal Methods and Performance Analysis.

[22]  David Blackwell,et al.  Positive dynamic programming , 1967 .

[23]  Martin Fränzle,et al.  Efficient Solving of Large Non-linear Arithmetic Constraint Systems with Complex Boolean Structure , 2007, J. Satisf. Boolean Model. Comput..

[24]  Lijun Zhang,et al.  Probabilistic reachability for parametric Markov models , 2010, International Journal on Software Tools for Technology Transfer.

[25]  Tingting Han,et al.  Diagnosis, Synthesis and Analysis of Probabilistic Models , 2009, Ausgezeichnete Informatikdissertationen.

[26]  Marta Z. Kwiatkowska,et al.  Automated Verification of a Randomized Distributed Consensus Protocol Using Cadence SMV and PRISM , 2001, CAV.

[27]  Rajeev Alur,et al.  A Temporal Logic of Nested Calls and Returns , 2004, TACAS.

[28]  Marta Z. Kwiatkowska,et al.  Stochastic Model Checking , 2007, SFM.

[29]  Andrea Maggiolo-Schettini,et al.  Parametric probabilistic transition systems for system design and analysis , 2007, Formal Aspects of Computing.

[30]  Leon G. Higley,et al.  Forensic Entomology: An Introduction , 2009 .

[31]  R. Strauch.  Negative Dynamic Programming , 1966 .

[32]  K. Siegrist.  How to Gamble If You Must , 2008 .

[33]  Bengt Jonsson,et al.  A logic for reasoning about time and reliability , 1990, Formal Aspects of Computing.

[34]  Paul B. Jackson,et al.  Combined Decision Techniques for the Existential Theory of the Reals , 2009, Calculemus/MKM.