Incremental quantitative verification for Markov decision processes

Quantitative verification techniques provide an effective means of computing performance and reliability properties for a wide range of systems. However, the computation required can be expensive, particularly if it has to be performed multiple times, for example to determine optimal system parameters. We present efficient incremental techniques for quantitative verification of Markov decision processes, which are able to re-use results from previous verification runs, based on a decomposition of the model into its strongly connected components (SCCs). We also show how this SCC-based approach can be further optimised to improve verification speed and how it can be combined with symbolic data structures to offer better scalability. We illustrate the effectiveness of the approach on a selection of large case studies.

[1]  Robert E. Tarjan,et al.  Depth-First Search and Linear Graph Algorithms , 1972, SIAM J. Comput..

[2]  Stephen A. Edwards,et al.  Incremental Algorithms for Inter-procedural Analysis of Safety Properties , 2005, CAV.

[3]  Radu Calinescu,et al.  Using quantitative analysis to implement autonomic IT systems , 2009, 2009 IEEE 31st International Conference on Software Engineering.

[4]  Marta Z. Kwiatkowska,et al.  Probabilistic Model Checking of the IEEE 802.11 Wireless Local Area Network Protocol , 2002, PAPM-PROBMIV.

[5]  Nils Jansen,et al.  DTMC Model Checking by SCC Reduction , 2010, 2010 Seventh International Conference on the Quantitative Evaluation of Systems.

[6]  Mihalis Yannakakis,et al.  The complexity of probabilistic verification , 1995, JACM.

[7]  Eljas Soisalon-Soininen,et al.  On Finding the Strongly Connected Components in a Directed Graph , 1994, Inf. Process. Lett..

[8]  Scott A. Smolka,et al.  Incremental Model Checking in the Modal Mu-Calculus , 1994, CAV.

[9]  Andrea Bianco,et al.  Model Checking of Probabalistic and Nondeterministic Systems , 1995, FSTTCS.

[10]  Christel Baier,et al.  Reduction Techniques for Model Checking Markov Decision Processes , 2008, 2008 Fifth International Conference on Quantitative Evaluation of Systems.

[11]  Marta Z. Kwiatkowska,et al.  Probabilistic Model Checking of Deadline Properties in the IEEE 1394 FireWire Root Contention Protocol , 2003, Formal Aspects of Computing.

[12]  Carla Piazza,et al.  Computing strongly connected components in a linear number of symbolic steps , 2003, SODA '03.

[13]  Fabio Somenzi,et al.  An Algorithm for Strongly Connected Component Analysis in n log n Symbolic Steps , 2000, Formal Methods Syst. Des..

[14]  Maurice Herlihy,et al.  Fast Randomized Consensus Using Shared Memory , 1990, J. Algorithms.

[15]  Timo Latvala,et al.  Incremental and Complete Bounded Model Checking for Full PLTL , 2005, CAV.

[16]  Marta Z. Kwiatkowska,et al.  PRISM 4.0: Verification of Probabilistic Real-Time Systems , 2011, CAV.

[17]  Fred Kröger,et al.  Temporal Logic of Programs , 1987, EATCS Monographs on Theoretical Computer Science.

[18]  Marta Z. Kwiatkowska,et al.  Performance analysis of probabilistic timed automata using digital clocks , 2003, Formal Methods Syst. Des..

[19]  Randal E. Bryant,et al.  Graph-Based Algorithms for Boolean Function Manipulation , 1986, IEEE Transactions on Computers.

[20]  J. Kemeny,et al.  Denumerable Markov chains , 1969 .

[21]  Martin L. Puterman,et al.  Markov Decision Processes: Discrete Stochastic Dynamic Programming , 1994 .

[22]  Zohar Manna,et al.  Formal verification of probabilistic systems , 1997 .

[23]  Edmund M. Clarke,et al.  Symbolic Model Checking: 10^20 States and Beyond , 1990, Inf. Comput..

[24]  Marta Z. Kwiatkowska,et al.  Automated Verification of a Randomized Distributed Consensus Protocol Using Cadence SMV and PRISM , 2001, CAV.

[25]  Carlo Ghezzi,et al.  Run-time efficient probabilistic model checking , 2011, 2011 33rd International Conference on Software Engineering (ICSE).

[26]  Bengt Jonsson,et al.  A logic for reasoning about time and reliability , 1990, Formal Aspects of Computing.