Compositional Bisimulation Minimization for Interval Markov Decision Processes

Formal verification of PCTL properties of MDPs with convex uncertainties has been recently investigated by Puggelli et al. However, model checking algorithms typically suffer from state space explosion. In this paper, we address probabilistic bisimulation to reduce the size of such an MDP while preserving PCTL properties it satisfies. We give a compositional reasoning over interval models to understand better the ways how large models with interval uncertainties can be composed. Afterwards, we discuss computational complexity of the bisimulation minimization and show that the problem is coNP-complete. Finally, we show that, under a mild condition, bisimulation can be computed in polynomial time.

[1]  Mayank Sharma,et al.  Supermodularity and Affine Policies in Dynamic Robust Optimization , 2013, Oper. Res..

[2]  Nadia Tawbi,et al.  Specification and Verification of the PowerScaleTM Bus Arbitration Protocol: An Industrial Experiment with LOTOS , 1996, FORTE.

[3]  Laurent El Ghaoui,et al.  Robust Control of Markov Decision Processes with Uncertain Transition Matrices , 2005, Oper. Res..

[4]  Daniel Kroening,et al.  Decision Procedures - An Algorithmic Point of View , 2008, Texts in Theoretical Computer Science. An EATCS Series.

[5]  P. Billingsley,et al.  Probability and Measure , 1980 .

[6]  Wang Yi,et al.  Algebraic Reasoning for Real-Time Probabilistic Processes with Uncertain Information , 1994, FTRTFT.

[7]  Kim Guldstrand Larsen,et al.  Specification and refinement of probabilistic processes , 1991, [1991] Proceedings Sixth Annual IEEE Symposium on Logic in Computer Science.

[8]  Ufuk Topcu,et al.  Robust control of uncertain Markov Decision Processes with temporal logic specifications , 2012, 2012 IEEE 51st IEEE Conference on Decision and Control (CDC).

[9]  Roberto Segala,et al.  Modeling and verification of randomized distributed real-time systems , 1996 .

[10]  Holger Hermanns,et al.  On the Efficiency of Deciding Probabilistic Automata Weak Bisimulation , 2013, Electron. Commun. Eur. Assoc. Softw. Sci. Technol..

[11]  K. Subramani,et al.  On quantified linear implications , 2014, Annals of Mathematics and Artificial Intelligence.

[12]  Kim G. Larsen,et al.  Decision Problems for Interval Markov Chains , 2011, LATA.

[13]  Robert E. Tarjan,et al.  Three Partition Refinement Algorithms , 1987, SIAM J. Comput..

[14]  Krishnendu Chatterjee,et al.  Model-Checking omega-Regular Properties of Interval Markov Chains , 2008, FoSSaCS.

[15]  Marta Z. Kwiatkowska,et al.  Stochastic Transition Systems for Continuous State Spaces and Non-determinism , 2005, FoSSaCS.

[16]  Roberto Segala,et al.  Decision Algorithms for Probabilistic Bisimulation , 2002, CONCUR.

[17]  Alexander Schrijver,et al.  Theory of linear and integer programming , 1986, Wiley-Interscience series in discrete mathematics and optimization.

[18]  Pravin M. Vaidya,et al.  An algorithm for linear programming which requires O(((m+n)n2+(m+n)1.5n)L) arithmetic operations , 1990, Math. Program..

[19]  Holger Hermanns,et al.  Towards Performance Prediction of Compositional Models in Industrial GALS Designs , 2009, CAV.

[20]  Jan Krcál,et al.  Probabilistic Bisimulations for PCTL Model Checking of Interval MDPs (extended version) , 2014, SynCoP.

[21]  Di Wu,et al.  Reachability analysis of uncertain systems using bounded-parameter Markov decision processes , 2008, Artif. Intell..

[22]  Joost-Pieter Katoen,et al.  Bisimulation Minimisation Mostly Speeds Up Probabilistic Model Checking , 2007, TACAS.

[23]  Mahesh Viswanathan,et al.  Model-Checking Markov Chains in the Presence of Uncertainties , 2006, TACAS.

[24]  Kim G. Larsen,et al.  Abstract Probabilistic Automata , 2011, VMCAI.

[25]  Joost-Pieter Katoen,et al.  Automated compositional Markov chain generation for a plain-old telephone system , 2000, Sci. Comput. Program..

[26]  K. Subramani,et al.  On the Complexities of Selected Satisfiability and Equivalence Queries over Boolean Formulas and Inclusion Queries over Hulls , 2009, Adv. Decis. Sci..

[27]  Taolue Chen,et al.  On the complexity of model checking interval-valued discrete time Markov chains , 2013, Inf. Process. Lett..

[28]  Garud Iyengar,et al.  Robust Dynamic Programming , 2005, Math. Oper. Res..

[29]  Holger Hermanns,et al.  Polynomial time decision algorithms for probabilistic automata , 2015, Inf. Comput..

[30]  Joost-Pieter Katoen,et al.  Compositional Abstraction for Stochastic Systems , 2009, FORMATS.

[31]  Lev V. Utkin,et al.  Interval-Valued Finite Markov Chains , 2002, Reliab. Comput..

[32]  Bengt Jonsson,et al.  A logic for reasoning about time and reliability , 1990, Formal Aspects of Computing.

[33]  David S. Johnson,et al.  Computers and Intractability: A Guide to the Theory of NP-Completeness , 1978 .

[34]  Lijun Zhang,et al.  Synthesis for PCTL in Parametric Markov Decision Processes , 2011, NASA Formal Methods.

[35]  Egon Balas,et al.  programming: Properties of the convex hull of feasible points * , 1998 .

[36]  Edward L. Braun,et al.  6 – Arithmetic Operations , 1963 .

[37]  Marta Z. Kwiatkowska,et al.  PRISM 4.0: Verification of Probabilistic Real-Time Systems , 2011, CAV.

[38]  Alberto L. Sangiovanni-Vincentelli,et al.  Polynomial-Time Verification of PCTL Properties of MDPs with Convex Uncertainties , 2013, CAV.

[39]  James Worrell,et al.  LTL Model Checking of Interval Markov Chains , 2013, TACAS.

[40]  Scott A. Smolka,et al.  CCS expressions, finite state processes, and three problems of equivalence , 1983, PODC '83.