A Modest Approach to Markov Automata

Markov automata are a compositional modelling formalism with continuous stochastic time, discrete probabilities, and nondeterministic choices. In this article, we present extensions to MODEST, an expressive high-level language with roots in process algebra, that allow large Markov automata models to be specified in a succinct, modular way. We illustrate the advantages of MODEST over alternative languages. Model checking Markov automata models requires dedicated algorithms for time-bounded and long-run average reward properties. We describe and evaluate the state-of-the-art algorithms implemented in the mcsta model checker of the MODEST TOOLSET. We find that mcsta improves the performance and scalability of Markov automata model checking compared to earlier and alternative tools. We explain a partial-exploration approach based on the BRTDP method designed to mitigate the state space explosion problem of model checking, and experimentally evaluate its effectiveness. This problem can be avoided entirely by purely simulation-based techniques, but the nondeterminism in Markov automata hinders their straightforward application. We explain how lightweight scheduler sampling can make simulation possible, and provide a detailed evaluation of its usefulness on several benchmarks using the MODEST TOOLSET’s modes simulator.

[1]  Kim G. Larsen,et al.  Uppaal Stratego , 2015, TACAS.

[2]  Lijun Zhang,et al.  A Semantics for Every GSPN , 2013, Petri Nets.

[3]  Mariëlle Stoelinga,et al.  Model-based testing of stochastically timed systems , 2019, Innovations in Systems and Software Engineering.

[4]  Holger Hermanns,et al.  Optimal Continuous Time Markov Decisions , 2015, ATVA.

[5]  Boudewijn R. Haverkort,et al.  On hypothesis testing for statistical model checking , 2015, International Journal on Software Tools for Technology Transfer.

[6]  Sven Schewe,et al.  Finite optimal control for time-bounded reachability in CTMDPs and continuous-time Markov games , 2010, Acta Informatica.

[7]  Joost-Pieter Katoen,et al.  Sound Value Iteration , 2018, CAV.

[8]  Holger Hermanns,et al.  Explicit Model Checking of Very Large MDP Using Partitioning and Secondary Storage , 2015, ATVA.

[9]  Pedro R. D'Argenio,et al.  Sampling Distributed Schedulers for Resilient Space Communication , 2020, NFM.

[10]  Joost-Pieter Katoen,et al.  A compositional modelling and analysis framework for stochastic hybrid systems , 2012, Formal Methods in System Design.

[11]  Robin Milner,et al.  Communication and concurrency , 1989, PHI Series in computer science.

[12]  David Parker,et al.  On Correctness, Precision, and Performance in Quantitative Verification - QComp 2020 Competition Report , 2020, ISoLA.

[13]  Mariëlle Stoelinga,et al.  Modelling and Analysis of Markov Reward Automata , 2014, ATVA.

[14]  Christel Baier,et al.  Performance evaluation and model checking join forces , 2010, Commun. ACM.

[15]  Holger Hermanns,et al.  The Modest Toolset: An Integrated Environment for Quantitative Modelling and Verification , 2014, TACAS.

[16]  Lijun Zhang,et al.  On Probabilistic Automata in Continuous Time , 2010, 2010 25th Annual IEEE Symposium on Logic in Computer Science.

[17]  Arnd Hartmanns,et al.  The Quantitative Verification Benchmark Set , 2019, TACAS.

[18]  Sean Sedwards,et al.  Efficient simulation-based verification of probabilistic timed automata , 2017, 2017 Winter Simulation Conference (WSC).

[19]  Kim G. Larsen,et al.  Time for Statistical Model Checking of Real-Time Systems , 2011, CAV.

[20]  Sebastian Junges,et al.  A Review of Statistical Model Checking Pitfalls on Real-Time Stochastic Models , 2014, ISoLA.

[21]  Sebastian Junges,et al.  A Storm is Coming: A Modern Probabilistic Model Checker , 2017, CAV.

[22]  Axel Legay,et al.  Smart sampling for lightweight verification of Markov decision processes , 2014, International Journal on Software Tools for Technology Transfer.

[23]  C. A. R. Hoare,et al.  Communicating sequential processes , 1978, CACM.

[24]  Geoffrey J. Gordon,et al.  Bounded real-time dynamic programming: RTDP with monotone upper bounds and performance guarantees , 2005, ICML.

[25]  Christian Georg Eisentraut,et al.  Principles of Markov automata , 2017 .

[26]  Salvatore J. Bavuso,et al.  Dynamic fault-tree models for fault-tolerant computer systems , 1992 .

[27]  Axel Legay,et al.  Statistical Approximation of Optimal Schedulers for Probabilistic Timed Automata , 2016, IFM.

[28]  Marco Beccuti,et al.  30 Years of GreatSPN , 2016 .

[29]  Roberto Segala,et al.  Modeling and verification of randomized distributed real-time systems , 1996 .

[30]  Christel Baier,et al.  Ensuring the Reliability of Your Model Checker: Interval Iteration for Markov Decision Processes , 2017, CAV.

[31]  Thomas A. Henzinger,et al.  Faster Statistical Model Checking for Unbounded Temporal Properties , 2016, TACAS.

[32]  Jan Kretínský,et al.  Monte Carlo Tree Search for Verifying Reachability in Markov Decision Processes , 2018, ISoLA.

[33]  Hassan Hatefi-Ardakani Finite horizon analysis of Markov automata , 2017 .

[34]  Axel Legay,et al.  Scalable Verification of Markov Decision Processes , 2013, SEFM Workshops.

[35]  Sean Sedwards,et al.  Lightweight Statistical Model Checking in Nondeterministic Continuous Time , 2018, ISoLA.

[36]  Gerardo Rubino,et al.  Rare Event Simulation using Monte Carlo Methods , 2009 .

[37]  Holger Hermanns,et al.  Continuous-Time Markov Decisions based on Partial Exploration , 2018, ATVA.

[38]  Holger Hermanns,et al.  Long-Run Rewards for Markov Automata , 2017, TACAS.

[39]  Sebastian Junges,et al.  JANI: Quantitative Model and Tool Interaction , 2017, TACAS.

[40]  Sebastian Junges,et al.  Markov automata with multiple objectives , 2017, Formal Methods in System Design.

[41]  Kim G. Larsen,et al.  Statistical Model Checking the 2018 Edition! , 2018, ISoLA.

[42]  Christel Baier,et al.  Model Checking Probabilistic Systems , 2018, Handbook of Model Checking.

[43]  Holger Hermanns,et al.  A Modest Approach to Modelling and Checking Markov Automata , 2019, QEST.

[44]  Benjamin Monmege,et al.  Interval iteration algorithm for MDPs and IMDPs , 2017, Theor. Comput. Sci..

[45]  Christel Baier,et al.  Principles of model checking , 2008 .

[46]  Olivier Buffet,et al.  Goal Probability Analysis in Probabilistic Planning: Exploring and Enhancing the State of the Art , 2016, J. Artif. Intell. Res..

[47]  Nick Hawes,et al.  Multi-Robot Planning Under Uncertain Travel Times and Safety Constraints , 2019, IJCAI.

[48]  Thomas Hérault,et al.  Approximate Probabilistic Model Checking , 2004, VMCAI.

[49]  Marta Z. Kwiatkowska,et al.  PRISM 4.0: Verification of Probabilistic Real-Time Systems , 2011, CAV.

[50]  Pavel Krcál,et al.  Scalable Analysis of Fault Trees with Dynamic Features , 2015, 2015 45th Annual IEEE/IFIP International Conference on Dependable Systems and Networks.

[51]  Joost-Pieter Katoen,et al.  Efficient Modelling and Generation of Markov Automata , 2012, CONCUR.

[52]  Tracy Camp,et al.  MANET simulation studies: the incredibles , 2005, MOCO.

[53]  Krishnendu Chatterjee,et al.  Verification of Markov Decision Processes Using Learning Algorithms , 2014, ATVA.

[54]  David Coppit,et al.  The Galileo fault tree analysis tool , 1999, Digest of Papers. Twenty-Ninth Annual International Symposium on Fault-Tolerant Computing (Cat. No.99CB36352).

[55]  Yishay Mansour,et al.  A Sparse Sampling Algorithm for Near-Optimal Planning in Large Markov Decision Processes , 1999, Machine Learning.

[56]  Pedro U. Lima,et al.  Long-Run Multi-Robot Planning With Uncertain Task Durations , 2020, AAMAS.

[57]  Joost-Pieter Katoen,et al.  Analysis of Timed and Long-Run Objectives for Markov Automata , 2014, Log. Methods Comput. Sci..

[58]  Blai Bonet,et al.  Labeled RTDP: Improving the Convergence of Real-Time Dynamic Programming , 2003, ICAPS.

[59]  Martin L. Puterman,et al.  Markov Decision Processes: Discrete Stochastic Dynamic Programming , 1994 .

[60]  Carlos E. Budde,et al.  An efficient statistical model checker for nondeterminism and rare events , 2020, International Journal on Software Tools for Technology Transfer.

[61]  Mariëlle Stoelinga,et al.  A Rigorous, Compositional, and Extensible Framework for Dynamic Fault Tree Analysis , 2010, IEEE Transactions on Dependable and Secure Computing.

[62]  Marta Z. Kwiatkowska,et al.  Automatic verification of real-time systems with discrete probability distributions , 1999, Theor. Comput. Sci..

[63]  Joost-Pieter Katoen,et al.  Quantitative Timed Analysis of Interactive Markov Chains , 2012, NASA Formal Methods.

[64]  Pedro R. D'Argenio,et al.  Automated compositional importance splitting , 2019, Sci. Comput. Program..

[65]  Holger Hermanns,et al.  Verification of Open Interactive Markov Chains , 2012, FSTTCS.

[66]  Holger Hermanns,et al.  MODEST: A Compositional Modeling Formalism for Hard and Softly Timed Systems , 2006, IEEE Transactions on Software Engineering.

[67]  Radu Mateescu,et al.  CADP 2011: a toolbox for the construction and analysis of distributed processes , 2012, International Journal on Software Tools for Technology Transfer.

[68]  Håkan L. S. Younes,et al.  Probabilistic Verification of Discrete Event Systems Using Acceptance Sampling , 2002, CAV.

[69]  Arnd Hartmanns,et al.  Optimistic Value Iteration , 2019, CAV.

[70]  Jan Kretínský,et al.  The 2019 Comparison of Tools for the Analysis of Quantitative Formal Models - (QComp 2019 Competition Report) , 2019, TACAS.

[71]  Hector Geffner,et al.  Heuristic Search for Generalized Stochastic Shortest Path MDPs , 2011, ICAPS.

[72]  Nataliya Sokolovska,et al.  Continuous Upper Confidence Trees , 2011, LION.

[73]  Yuliya Butkova,et al.  Optimal Time-Bounded Reachability Analysis for Concurrent Systems , 2019, TACAS.

[74]  Simon M. Lucas,et al.  A Survey of Monte Carlo Tree Search Methods , 2012, IEEE Transactions on Computational Intelligence and AI in Games.

[75]  Holger Hermanns,et al.  A Modest Markov Automata Tutorial , 2019, RW.

[76]  Holger Hermanns,et al.  Markov Automata on Discount! , 2018, MMB.