The Probabilistic Model Checking Landscape*

Randomization is a key element in sequential and distributed computing. Reasoning about randomized algorithms is highly non-trivial. In the 1980s, this initiated first proof methods, logics, and model-checking algorithms. The field of probabilistic verification has developed considerably since then. This paper surveys the algorithmic verification of probabilistic models, in particular probabilistic model checking. We provide an informal account of the main models, the underlying algorithms, applications from reliability and dependability analysis—and beyond—and describe recent developments towards automated parameter synthesis.

[1]  Kousha Etessami,et al.  Multi-objective Model Checking of Markov Decision Processes , 2007, TACAS.

[2]  Moshe Y. Vardi Automatic verification of probabilistic concurrent finite state programs , 1985, 26th Annual Symposium on Foundations of Computer Science (sfcs 1985).

[3]  Erik P. de Vink,et al.  A hierarchy of probabilistic system types , 2003, CMCS.

[4]  Holger Hermanns,et al.  In the quantitative automata zoo , 2015, Sci. Comput. Program..

[5]  Diego Latella,et al.  Continuous approximation of collective system behaviour: A tutorial , 2013, Perform. Evaluation.

[6]  Gunter Bolch,et al.  Queueing Networks and Markov Chains - Modeling and Performance Evaluation with Computer Science Applications, Second Edition , 1998 .

[7]  Peter Buchholz,et al.  Numerical analysis of continuous time Markov decision processes over finite horizons , 2011, Comput. Oper. Res..

[8]  Christel Baier,et al.  Ratio and Weight Quantiles , 2015, MFCS.

[9]  Edmund M. Clarke,et al.  The Birth of Model Checking , 2008, 25 Years of Model Checking.

[10]  Eric Klavins,et al.  A compositional framework for programming stochastically interacting robots , 2011, Int. J. Robotics Res..

[11]  Joost-Pieter Katoen,et al.  Automated compositional Markov chain generation for a plain-old telephone system , 2000, Sci. Comput. Program..

[12]  Joost-Pieter Katoen,et al.  Approximate Model Checking of Stochastic Hybrid Systems , 2010, Eur. J. Control.

[13]  Joost-Pieter Katoen Model Checking Meets Probability: A Gentle Introduction , 2013, Engineering Dependable Software Systems.

[14]  Axel Legay,et al.  Statistical Model Checking: An Overview , 2010, RV.

[15]  Nancy A. Lynch,et al.  Switched PIOA: Parallel composition via distributed scheduling , 2006, Theor. Comput. Sci..

[16]  Donald Gross,et al.  The Randomization Technique as a Modeling Tool and Solution Procedure for Transient Markov Processes , 1984, Oper. Res..

[17]  M.D. Beaudry,et al.  PERFORMANCE RELATED RELIABILITY MEASURES FOR COMPUTING SYSTEMS , 1995, Twenty-Fifth International Symposium on Fault-Tolerant Computing, 1995, ' Highlights from Twenty-Five Years'..

[18]  Flemming Nielson,et al.  Automata-Based CSL Model Checking , 2011, ICALP.

[19]  Marco Ajmone Marsan,et al.  A class of generalized stochastic Petri nets for the performance evaluation of multiprocessor systems , 1984, TOCS.

[20]  Nils Jansen,et al.  Counterexample Generation for Discrete-Time Markov Models: An Introductory Survey , 2014, SFM.

[21]  Kim G. Larsen,et al.  Bisimulation through Probabilistic Testing , 1991, Inf. Comput..

[22]  Chris Thachuk,et al.  Probabilistic Model Checking for Biology , 2014, Software Systems Safety.

[23]  Micha Sharir,et al.  Termination of Probabilistic Concurrent Program , 1983, TOPL.

[24]  Lijun Zhang,et al.  Best Probabilistic Transformers , 2010, VMCAI.

[25]  Sebastian Junges,et al.  SMT-RAT: An Open Source C++ Toolbox for Strategic and Parallel SMT Solving , 2015, SAT.

[26]  Kousha Etessami,et al.  Analysis of probabilistic processes and automata theory , 2021, Handbook of Automata Theory.

[27]  Thomas A. Henzinger,et al.  Theory in practice for system design and verification , 2015, SIGL.

[28]  Marco Beccuti,et al.  30 Years of GreatSPN , 2016 .

[29]  Sriram K. Rajamani,et al.  Efficient synthesis of probabilistic programs , 2015, PLDI.

[30]  Bernd Becker,et al.  Compositional Dependability Evaluation for STATEMATE , 2009, IEEE Transactions on Software Engineering.

[31]  James Worrell,et al.  On the Complexity of Computing Probabilistic Bisimilarity , 2012, FoSSaCS.

[32]  Christel Baier,et al.  LiQuor: A tool for Qualitative and Quantitative Linear Time analysis of Reactive Systems , 2006, Third International Conference on the Quantitative Evaluation of Systems - (QEST'06).

[33]  Christel Baier,et al.  Model checking probabilistic systems against pushdown specifications , 2012, Inf. Process. Lett..

[34]  Mihalis Yannakakis,et al.  Verifying temporal properties of finite-state probabilistic programs , 1988, [Proceedings 1988] 29th Annual Symposium on Foundations of Computer Science.

[35]  David S. Rosenblum,et al.  Perturbation Analysis in Verification of Discrete-Time Markov Chains , 2014, CONCUR.

[36]  Parosh Aziz Abdulla,et al.  Decisive Markov Chains , 2007, Log. Methods Comput. Sci..

[37]  Bengt Jonsson,et al.  A logic for reasoning about time and reliability , 1990, Formal Aspects of Computing.

[38]  Philippe Schnoebelen,et al.  The Verification of Probabilistic Lossy Channel Systems , 2004, Validation of Stochastic Systems.

[39]  Anja Vogler,et al.  Continuous Time Markov Decision Processes Theory And Applications , 2016 .

[40]  Peter C. Kiessler,et al.  A critical look at the bathtub curve , 2003, IEEE Trans. Reliab..

[41]  Mickael Randour,et al.  Percentile queries in multi-dimensional Markov decision processes , 2014, CAV.

[42]  Lijun Zhang,et al.  Probably safe or live , 2014, CSL-LICS.

[43]  Sebastian Junges,et al.  Parameter Synthesis for Markov Models: Faster Than Ever , 2016, ATVA.

[44]  Lijun Zhang,et al.  On Probabilistic Automata in Continuous Time , 2010, 2010 25th Annual IEEE Symposium on Logic in Computer Science.

[45]  Joost-Pieter Katoen,et al.  A Model Checker for AADL , 2010, CAV.

[46]  Joost-Pieter Katoen,et al.  Discrete-Time Rewards Model-Checked , 2003, FORMATS.

[47]  Daniel Gebler,et al.  On Abstraction of Probabilistic Systems , 2012, ROCKS.

[48]  Luca de Alfaro,et al.  How to Specify and Verify the Long-Run Average Behavior of Probabilistic Systems , 1998, LICS.

[49]  Makis Stamatelatos,et al.  Fault tree handbook with aerospace applications , 2002 .

[50]  Martin L. Puterman,et al.  Markov Decision Processes: Discrete Stochastic Dynamic Programming , 1994 .

[51]  Yi Li,et al.  A Web-Based Probabilistic Model Checker ? , 2014 .

[52]  Marta Z. Kwiatkowska,et al.  Automated Verification Techniques for Probabilistic Systems , 2011, SFM.

[53]  Dragan Bosnacki,et al.  Parallel probabilistic model checking on general purpose graphics processors , 2010, International Journal on Software Tools for Technology Transfer.

[54]  Monika Heiner,et al.  Petri Nets for Systems and Synthetic Biology , 2008, SFM.

[55]  Joost-Pieter Katoen,et al.  The Ins and Outs of the Probabilistic Model Checker MRMC , 2009, 2009 Sixth International Conference on the Quantitative Evaluation of Systems.

[56]  Verena Wolf,et al.  On-the-fly verification and optimization of DTA-properties for large Markov chains , 2012, Formal Methods in System Design.

[57]  Javier Esparza,et al.  Analyzing probabilistic pushdown automata , 2013, Formal Methods Syst. Des..

[58]  Ronald A. Howard,et al.  Dynamic Probabilistic Systems , 1971 .

[59]  D. Vere-Jones Markov Chains , 1972, Nature.

[60]  Joost-Pieter Katoen,et al.  Spacecraft early design validation using formal methods , 2014, Reliab. Eng. Syst. Saf..

[61]  Thomas A. Henzinger,et al.  Equivalence of Labeled Markov Chains , 2008, Int. J. Found. Comput. Sci..

[62]  Luca Cardelli,et al.  Design and analysis of DNA strand displacement devices using probabilistic model checking , 2012, Journal of The Royal Society Interface.

[63]  Matthew Chalmers,et al.  Probabilistic Formal Analysis of App Usage to Inform Redesign , 2015, IFM.

[64]  Nancy A. Lynch,et al.  Probabilistic Simulations for Probabilistic Processes , 1994, Nord. J. Comput..

[65]  Edmund M. Clarke,et al.  Assume-Guarantee Abstraction Refinement for Probabilistic Systems , 2012, CAV.

[66]  J. Lisman,et al.  Note on the generation of most probable frequency distributions , 1972 .

[67]  Christel Baier,et al.  Reachability in continuous-time Markov reward decision processes , 2008, Logic and Automata.

[68]  Joël Ouaknine,et al.  Reachability problems for Markov chains , 2015, Inf. Process. Lett..

[69]  Joost-Pieter Katoen,et al.  Time-bounded reachability in tree-structured QBDs by abstraction , 2011, Perform. Evaluation.

[70]  Joost-Pieter Katoen,et al.  Safety, Dependability and Performance Analysis of Extended AADL Models , 2011, Comput. J..

[71]  Christel Baier,et al.  Controller Synthesis for Probabilistic Systems , 2004, IFIP TCS.

[72]  Prakash Panangaden,et al.  Labelled Markov Processes , 2009 .

[73]  Joost-Pieter Katoen,et al.  A theory of stochastic systems part I: Stochastic automata , 2005, Inf. Comput..

[74]  C. Loan,et al.  Nineteen Dubious Ways to Compute the Exponential of a Matrix , 1978 .

[75]  Christel Baier,et al.  Computing Conditional Probabilities in Markovian Models Efficiently , 2014, TACAS.

[76]  Giuliana Franceschinis,et al.  Simple O(m logn) Time Markov Chain Lumping , 2010, TACAS.

[77]  John G. Kemeny,et al.  Finite Markov chains , 1960 .

[78]  Giordano Tamburrelli,et al.  Reliability of Run-Time Quality-of-Service Evaluation Using Parametric Model Checking , 2016, 2016 IEEE/ACM 38th International Conference on Software Engineering (ICSE).

[79]  Azaria Paz,et al.  Probabilistic automata , 2003 .

[80]  Andrea Maggiolo-Schettini,et al.  Parametric probabilistic transition systems for system design and analysis , 2007, Formal Aspects of Computing.

[81]  Nikolaj Bjørner,et al.  Z3: An Efficient SMT Solver , 2008, TACAS.

[82]  Mariëlle Stoelinga,et al.  A Rigorous, Compositional, and Extensible Framework for Dynamic Fault Tree Analysis , 2010, IEEE Transactions on Dependable and Secure Computing.

[83]  Christel Baier,et al.  Partial Order Reduction for Probabilistic Branching Time , 2006, QAPL.

[84]  Sandeep K. Shukla,et al.  Using probabilistic model checking for dynamic power management , 2005, Formal Aspects of Computing.

[85]  Kim G. Larsen,et al.  Uppaal SMC tutorial , 2015, International Journal on Software Tools for Technology Transfer.

[86]  Mariëlle Stoelinga,et al.  Fault tree analysis: A survey of the state-of-the-art in modeling, analysis and tools , 2014, Comput. Sci. Rev..

[87]  Hongyang Qu,et al.  Compositional probabilistic verification through multi-objective model checking , 2013, Inf. Comput..

[88]  Nils Jansen,et al.  Accelerating Parametric Probabilistic Verification , 2014, QEST.

[89]  Joost-Pieter Katoen,et al.  Efficient CTMC Model Checking of Linear Real-Time Objectives , 2011, TACAS.

[90]  Antonín Kucera,et al.  On the Controller Synthesis for Finite-State Markov Decision Processes , 2005, Fundam. Informaticae.

[91]  Krishnendu Chatterjee,et al.  Measuring and Synthesizing Systems in Probabilistic Environments , 2010, CAV.

[92]  Christel Baier,et al.  Principles of model checking , 2008 .

[93]  B. L. Miller Finite State Continuous Time Markov Decision Processes with a Finite Planning Horizon , 1968 .

[94]  Christel Baier,et al.  Efficient computation of time-bounded reachability probabilities in uniform continuous-time Markov decision processes , 2005, Theor. Comput. Sci..

[95]  Christel Baier,et al.  Probabilistic Model Checking , 2016, Dependable Software Systems Engineering.

[96]  J. Filar,et al.  Competitive Markov Decision Processes , 1996 .

[97]  Joël Ouaknine,et al.  APEX: An Analyzer for Open Probabilistic Programs , 2012, CAV.

[98]  Joost-Pieter Katoen,et al.  Probabilistic Model Checking for Uncertain Scenario-Aware Data Flow , 2016, TODE.

[99]  Nicola Paoletti,et al.  Precise Parameter Synthesis for Stochastic Biochemical Systems , 2014, CMSB.

[100]  Krishnendu Chatterjee,et al.  The Complexity of Synthesis from Probabilistic Components , 2015, ICALP.

[101]  Xianping Guo,et al.  Continuous-Time Markov Decision Processes: Theory and Applications , 2009 .

[102]  William H. Sanders,et al.  State-Space Support for Path-Based Reward Variables , 1999, Perform. Evaluation.

[103]  Marta Z. Kwiatkowska,et al.  PRISM 4.0: Verification of Probabilistic Real-Time Systems , 2011, CAV.

[104]  W E Vesely,et al.  Fault Tree Handbook , 1987 .

[105]  Christel Baier,et al.  Model-Checking Algorithms for , 2003 .

[106]  Christel Baier,et al.  Stochastic Timed Automata , 2014, Log. Methods Comput. Sci..

[107]  Dejan Jovanović,et al.  Solving Non-linear Arithmetic , 2012, IJCAR.

[108]  Kim G. Larsen,et al.  Abstract Probabilistic Automata , 2011, VMCAI.

[109]  Christel Baier,et al.  Principles of Model Checking (Representation and Mind Series) , 2008 .

[110]  Joost-Pieter Katoen,et al.  Formal correctness, safety, dependability, and performance analysis of a satellite , 2012, 2012 34th International Conference on Software Engineering (ICSE).

[111]  Joost-Pieter Katoen,et al.  Compositional Abstraction for Stochastic Systems , 2009, FORMATS.

[112]  Carlo Ghezzi,et al.  Self-adaptive software needs quantitative verification at runtime , 2012, CACM.

[113]  Marta Z. Kwiatkowska,et al.  Symbolic model checking for probabilistic timed automata , 2007, Inf. Comput..

[114]  Marta Z. Kwiatkowska,et al.  A game-based abstraction-refinement framework for Markov decision processes , 2010, Formal Methods Syst. Des..

[115]  Holger Hermanns,et al.  Interactive Markov Chains , 2002, Lecture Notes in Computer Science.

[116]  Conrado Daws Symbolic and Parametric Model Checking of Discrete-Time Markov Chains , 2004, ICTAC.

[117]  Marta Z. Kwiatkowska,et al.  PRISM-Games 2.0: A Tool for Multi-objective Strategy Synthesis for Stochastic Games , 2016, TACAS.

[118]  J. Kemeny,et al.  Denumerable Markov chains , 1969 .

[119]  Peter H. Feiler,et al.  Model-Based Engineering with AADL: An Introduction to the SAE Architecture Analysis & Design Language , 2012 .

[120]  John G. Kemeny,et al.  Finite Markov Chains. , 1960 .

[121]  Christel Baier,et al.  Probabilistic ω-automata , 2012, JACM.

[122]  Joost-Pieter Katoen,et al.  Quantitative Model Checking of Continuous-Time Markov Chains Against Timed Automata Specifications , 2009, 2009 24th Annual IEEE Symposium on Logic In Computer Science.

[123]  Lijun Zhang,et al.  A Semantics for Every GSPN , 2013, Petri Nets.

[124]  Christel Baier,et al.  Symbolic Model Checking for Probabilistic Processes , 1997, ICALP.

[125]  Amir Pnueli,et al.  Probabilistic Verification by Tableaux , 1986, Logic in Computer Science.

[126]  Christel Baier,et al.  Model checking for a probabilistic branching time logic with fairness , 1998, Distributed Computing.

[127]  Dexter Kozen,et al.  A probabilistic PDL , 1983, J. Comput. Syst. Sci..

[128]  Jane Hillston,et al.  Process algebras for quantitative analysis , 2005, 20th Annual IEEE Symposium on Logic in Computer Science (LICS' 05).

[129]  Gethin Norman,et al.  Model checking for probabilistic timed automata , 2012, Formal Methods in System Design.

[130]  Christel Baier,et al.  Model-Checking Algorithms for Continuous-Time Markov Chains , 2002, IEEE Trans. Software Eng..

[131]  Erik P. de Vink,et al.  A hierarchy of probabilistic system types , 2003, CMCS.

[132]  Mihalis Yannakakis,et al.  The complexity of probabilistic verification , 1995, JACM.

[133]  James Worrell,et al.  A behavioural pseudometric for probabilistic transition systems , 2005, Theor. Comput. Sci..

[134]  Mariëlle Stoelinga,et al.  An Introduction to Probabilistic Automata , 2002, Bull. EATCS.

[135]  Serge Haddad,et al.  Model Checking Timed and Stochastic Properties with CSL^{TA} , 2009, IEEE Transactions on Software Engineering.

[136]  Joost-Pieter Katoen,et al.  Parametric LTL on Markov Chains , 2014, IFIP TCS.

[137]  Christel Baier,et al.  Performability assessment by model checking of Markov reward models , 2010, Formal Methods Syst. Des..

[138]  Sebastian Junges,et al.  PROPhESY: A PRObabilistic ParamEter SYnthesis Tool , 2015, CAV.

[139]  Scott A. Smolka,et al.  Composition and Behaviors of Probabilistic I/O Automata , 1994, Theor. Comput. Sci..

[140]  Véronique Bruyère,et al.  Meet Your Expectations With Guarantees: Beyond Worst-Case Synthesis in Quantitative Games , 2013, STACS.

[141]  Marcel F. Neuts,et al.  Matrix-geometric solutions in stochastic models - an algorithmic approach , 1982 .

[142]  Matthew Hennessy,et al.  On the semantics of Markov automata , 2011, Inf. Comput..

[143]  Joost-Pieter Katoen,et al.  Faster and Symbolic CTMC Model Checking , 2001, PAPM-PROBMIV.

[144]  Robert K. Brayton,et al.  Model-checking continuous-time Markov chains , 2000, TOCL.

[145]  Vitaly Shmatikov,et al.  Analysis of probabilistic contract signing , 2002, J. Comput. Secur..

[146]  Azaria Paz,et al.  Introduction to probabilistic automata (Computer science and applied mathematics) , 1971 .

[147]  Luca Pulina,et al.  Evaluating probabilistic model checking tools for verification of robot control policies , 2016, AI Commun..

[148]  B. L. Miller Finite state continuous time Markov decision processes with an infinite planning horizon , 1968 .

[149]  Gethin Norman,et al.  Analysing Randomized Distributed Algorithms , 2004, Validation of Stochastic Systems.

[150]  Joost-Pieter Katoen,et al.  Three-valued abstraction for probabilistic systems , 2012, J. Log. Algebraic Methods Program..

[151]  Christel Baier,et al.  Trade-off analysis meets probabilistic model checking , 2014, CSL-LICS.

[152]  Dexter Kozen,et al.  Semantics of probabilistic programs , 1979, 20th Annual Symposium on Foundations of Computer Science (sfcs 1979).

[153]  Gunter Bolch,et al.  Queueing Networks and Markov Chains , 2005 .

[154]  Christel Baier,et al.  Comparative branching-time semantics for Markov chains , 2005, Inf. Comput..

[155]  Scott A. Smolka,et al.  Compositional analysis of expected delays in networks of probabilistic I/O automata , 1998, Proceedings. Thirteenth Annual IEEE Symposium on Logic in Computer Science (Cat. No.98CB36226).

[156]  Chris Thachuk,et al.  DNA walker circuits: computational potential, design, and verification , 2013, Natural Computing.

[157]  Zhen Zhang,et al.  Stochastic Model Checking of Genetic Circuits , 2014, ACM J. Emerg. Technol. Comput. Syst..

[158]  Joost-Pieter Katoen,et al.  Zero-reachability in probabilistic multi-counter automata , 2014, CSL-LICS.

[159]  Joost-Pieter Katoen,et al.  Delayed Nondeterminism in Continuous-Time Markov Decision Processes , 2009, FoSSaCS.

[160]  Holger Hermanns,et al.  Optimal Continuous Time Markov Decisions , 2015, ATVA.

[161]  C. R. Ramakrishnan,et al.  Model Repair for Probabilistic Systems , 2011, TACAS.

[162]  Manfred Kudlek,et al.  Probability in Petri Nets , 2005, Fundam. Informaticae.

[163]  Lijun Zhang,et al.  Efficient approximation of optimal control for continuous-time Markov games , 2010, Inf. Comput..

[164]  Joseph F. Traub,et al.  Algorithms and Complexity: New Directions and Recent Results , 1976 .

[165]  Cleve B. Moler,et al.  Nineteen Dubious Ways to Compute the Exponential of a Matrix, Twenty-Five Years Later , 1978, SIAM Rev..

[166]  Andrea Bianco,et al.  Model Checking of Probabalistic and Nondeterministic Systems , 1995, FSTTCS.

[167]  Yi Li,et al.  iscasMc: A Web-Based Probabilistic Model Checker , 2014, FM.

[168]  Joost-Pieter Katoen,et al.  Quantitative Model Checking of Continuous-Time Markov Chains Against Timed Automata Specifications , 2009, 2009 24th Annual IEEE Symposium on Logic In Computer Science.

[169]  Krishnendu Chatterjee,et al.  A survey of stochastic ω-regular games , 2012, J. Comput. Syst. Sci..

[170]  Marta Z. Kwiatkowska,et al.  Symmetry Reduction for Probabilistic Model Checking , 2006, CAV.

[171]  Håkan L. S. Younes,et al.  Statistical probabilistic model checking with a focus on time-bounded properties , 2006, Inf. Comput..

[172]  Salvatore J. Bavuso,et al.  Dynamic fault-tree models for fault-tolerant computer systems , 1992 .

[173]  Joost P. Katoen,et al.  Smart railroad maintenance engineering with stochastic model checking , 2014 .

[174]  Yoad Lustig,et al.  Synthesis from Probabilistic Components , 2014, CSL.

[175]  Jun Sun,et al.  A Model Checker for Hierarchical Probabilistic Real-Time Systems , 2012, CAV.

[176]  Joost-Pieter Katoen,et al.  Analysis of Timed and Long-Run Objectives for Markov Automata , 2014, Log. Methods Comput. Sci..

[177]  Joost-Pieter Katoen,et al.  Approximate Parameter Synthesis for Probabilistic Time-Bounded Reachability , 2008, 2008 Real-Time Systems Symposium.

[178]  Christel Baier,et al.  Model checking for performability , 2013, Mathematical Structures in Computer Science.

[179]  Lijun Zhang,et al.  Probabilistic reachability for parametric Markov models , 2010, International Journal on Software Tools for Technology Transfer.

[180]  Sebastian Junges,et al.  Advancing Dynamic Fault Tree Analysis , 2016, ArXiv.

[181]  Kousha Etessami,et al.  Recursive Markov Decision Processes and Recursive Stochastic Games , 2005, ICALP.

[182]  Fan Long,et al.  Automatic patch generation by learning correct code , 2016, POPL.

[183]  Joost-Pieter Katoen,et al.  Bisimulation Minimisation Mostly Speeds Up Probabilistic Model Checking , 2007, TACAS.

[184]  Hongfei Fu,et al.  Maximal Cost-Bounded Reachability Probability on Continuous-Time Markov Decision Processes , 2013, FoSSaCS.

[185]  Marta Z. Kwiatkowska,et al.  Stochastic Model Checking , 2007, SFM.