A Review of Statistical Model Checking Pitfalls on Real-Time Stochastic Models

Statistical model checking (SMC) is a technique inspired by Monte-Carlo simulation for verifying time-bounded temporal logical properties. SMC originally focused on fully stochastic models such as Markov chains, but its scope has recently been extended to cover formalisms that mix functional real-time aspects, concurrency and non-determinism. We show by various examples using the tools UPPAAL SMC and Modes that combining the stochastic interpretation of such models with SMC algorithms is extremely subtle. This may yield significant discrepancies in the analysis results. As these subtleties are not so obvious to the end-user, we present five semantic caveats and give a classification scheme for SMC algorithms. We argue that caution is needed and believe that the caveats and classification scheme in this paper serve as a guiding reference for thoroughly understanding them.

[1]  Holger Hermanns,et al.  Partial Order Methods for Statistical Model Checking and Simulation , 2011, FMOODS/FORTE.

[2]  Cyrille Jégourel,et al.  A Platform for High Performance Statistical Model Checking - PLASMA , 2012, TACAS.

[3]  Holger Hermanns,et al.  MODEST: A Compositional Modeling Formalism for Hard and Softly Timed Systems , 2006, IEEE Transactions on Software Engineering.

[4]  Joost-Pieter Katoen,et al.  A probabilistic extension of UML statecharts: Specification and Verification. , 2002 .

[5]  Sanjiva Prasad,et al.  FSTTCS 2007: Foundations of Software Technology and Theoretical Computer Science, 27th International Conference, New Delhi, India, December 12-14, 2007, Proceedings , 2007, FSTTCS.

[6]  Radu Grosu,et al.  Monte Carlo Model Checking , 2005, TACAS.

[7]  Axel Legay,et al.  Statistical Model Checking: An Overview , 2010, RV.

[8]  Arnd Hartmanns,et al.  On-the-Fly Confluence Detection for Statistical Model Checking , 2013, NASA Formal Methods.

[9]  Kim G. Larsen,et al.  Memory Efficient Data Structures for Explicit Verification of Timed Systems , 2014, NASA Formal Methods.

[10]  Christel Baier,et al.  Probabilistic and Topological Semantics for Timed Automata , 2007, FSTTCS.

[11]  Lijun Zhang,et al.  Measurability and safety verification for stochastic hybrid systems , 2011, HSCC '11.

[12]  Stephen Gilmore,et al.  Specifying Performance Measures for PEPA , 1999, ARTS.

[13]  Kim G. Larsen,et al.  Minimum-Cost Reachability for Priced Timed Automata , 2001, HSCC.

[14]  Rajeev Alur,et al.  A Theory of Timed Automata , 1994, Theor. Comput. Sci..

[15]  George J. Pappas,et al.  Optimal Paths in Weighted Timed Automata , 2001, HSCC.

[16]  Rajeev Alur,et al.  A Temporal Logic of Nested Calls and Returns , 2004, TACAS.

[17]  Holger Hermanns,et al.  Interactive Markov Chains , 2002, Lecture Notes in Computer Science.

[18]  Axel Legay,et al.  PLASMA-lab: A Flexible, Distributable Statistical Model Checking Library , 2013, QEST.

[19]  Håkan L. S. Younes,et al.  Statistical probabilistic model checking with a focus on time-bounded properties , 2006, Inf. Comput..

[20]  Kim G. Larsen,et al.  UPPAAL-SMC: Statistical Model Checking for Priced Timed Automata , 2012, QAPL.

[21]  Klaus Echtle,et al.  Measurement, Modelling, and Evaluation of Computing Systems and Dependability and Fault Tolerance, 15th International GI/ITG Conference, MMB&DFT 2010, Essen, Germany, March 15-17, 2010. Proceedings , 2010, MMB/DFT.

[22]  Christel Baier,et al.  Model-Checking Algorithms for Continuous-Time Markov Chains , 2002, IEEE Trans. Software Eng..

[23]  Thomas A. Henzinger,et al.  Hybrid Systems: Computation and Control , 1998, Lecture Notes in Computer Science.

[24]  Jeremy Sproston.  Decidable Model Checking of Probabilistic Hybrid Automata , 2000, FTRTFT.

[25]  David Lee,et al.  Formal Techniques for Distributed Systems, Joint 11th IFIP WG 6.1 International Conference FMOODS 2009 and 29th IFIP WG 6.1 International Conference FORTE 2009, Lisboa, Portugal, June 9-12, 2009. Proceedings , 2009, FMOODS/FORTE.

[26]  Holger Hermanns,et al.  Simulation and Statistical Model Checking for Modestly Nondeterministic Models , 2012, MMB/DFT.

[27]  Jane Hillston,et al.  PALOMA: A Process Algebra for Located Markovian Agents , 2014, QEST.

[28]  Marco Ajmone Marsan,et al.  The Effect of Execution Policies on the Semantics and Analysis of Stochastic Petri Nets , 1989, IEEE Trans. Software Eng..

[29]  Stavros Tripakis,et al.  Verifying Progress in Timed Systems , 1999, ARTS.