Modeling and Verification of Distributed Systems Using Markov Decision Processes

The Markov Decision Process (MDP) formalism is a well-known mathematical formalism to study systems with unknown scheduling mechanisms or with transitions whose next-state probability distribution is not known with precision. Analysis methods for MDPs are based generally on the identification of the strategies that maximize (or minimize) a target function based on the MDP’s rewards (or costs). Alternatively, formal languages can be defined to express quantitative properties that we want to be ensured by an MDP, including those which extend classical temporal logics with probabilistic operators. Dario Bruneo and Salvatore Distefano (eds.) Quantitative Assessments of Distributed Systems, (3–26) 2015 © Scrivener Publishing LLC 3 4 Quantitative Assessments of Distributed Systems The MDP formalism is low level: to facilitate the representation of complex reallife distributed systems higher-level languages have been proposed. In this chapter we consider Markov Decision Well-formed Nets (MDWN), which are probabilistic extensions of Petri nets that allow one to describe complex nondeterministic (probabilistic) behavior as a composition of simpler nondeterministic (probabilistic) steps, and which inherit the efficient analysis algorithms originally devised for well-formed Petri nets. The features of the formalism and the type of properties that can be studied are illustrated by an example of a peer-to-peer illegal botnet.

[1]  Marta Z. Kwiatkowska,et al.  Symmetry Reduction for Probabilistic Model Checking , 2006, CAV.

[2]  R. Bellman.  Dynamic programming. , 1957, Science.

[3]  Wang Yi,et al.  Probabilistic Extensions of Process Algebras , 2001, Handbook of Process Algebra.

[4]  Christel Baier,et al.  Model checking for a probabilistic branching time logic with fairness , 1998, Distributed Computing.

[5]  Andrea Bianco,et al.  Model Checking of Probabalistic and Nondeterministic Systems , 1995, FSTTCS.

[6]  Edmund M. Clarke,et al.  Model Checking , 1999, Handbook of Automated Reasoning.

[7]  Ronald A. Howard,et al.  Dynamic Programming and Markov Processes , 1960 .

[8]  Marta Z. Kwiatkowska,et al.  Automated Verification Techniques for Probabilistic Systems , 2011, SFM.

[9]  Thomas A. Henzinger,et al.  Reactive Modules , 1999, Formal Methods Syst. Des..

[10]  Michael P. Wellman,et al.  Planning and Control , 1991 .

[11]  Bengt Jonsson,et al.  A logic for reasoning about time and reliability , 1990, Formal Aspects of Computing.

[12]  William H. Sanders,et al.  Modeling Peer-to-Peer Botnets , 2008, 2008 Fifth International Conference on Quantitative Evaluation of Systems.

[13]  Leslie Pack Kaelbling,et al.  On the Complexity of Solving Markov Decision Problems , 1995, UAI.

[14]  Christel Baier,et al.  Principles of model checking , 2008 .

[15]  Marta Z. Kwiatkowska,et al.  PRISM 4.0: Verification of Probabilistic Real-Time Systems , 2011, CAV.

[16]  Luca de Alfaro,et al.  Stochastic Transition Systems , 1998, CONCUR.

[17]  Holger Hermanns,et al.  MODEST: A Compositional Modeling Formalism for Hard and Softly Timed Systems , 2006, IEEE Transactions on Software Engineering.

[18]  Giovanni Chiola,et al.  Stochastic Well-Formed Colored Nets and Symmetric Modeling Applications , 1993, IEEE Trans. Computers.

[19]  Marco Beccuti,et al.  Markov Decision Petri Net and Markov Decision Well-Formed Net Formalisms , 2007, ICATPN.

[20]  Amir Pnueli,et al.  The temporal logic of programs , 1977, 18th Annual Symposium on Foundations of Computer Science (sfcs 1977).

[21]  Marco Beccuti,et al.  MDWNsolver: A Framework to Design and Solve Markov Decision Petri Nets , 2011 .