Safety Verification of Continuous-Space Pure Jump Markov Processes

We study the probabilistic safety verification problem for pure jump Markov processes, a class of models that generalizes continuous-time Markov chains over continuous uncountable state spaces. Solutions of these processes are piecewise constant, right-continuous functions from time to states. Their jump or reset times are realizations of a Poisson process, characterized by a jump rate function that can be both time- and state-dependent. Upon jumping in time, the new state of the solution process is specified according to a continuous stochastic conditional kernel. After providing a full characterization of safety properties of these processes, we describe a formal method to abstract the process as a finite-state discrete-time Markov chain; this approach is formal in that it provides a-priori error bounds on the precision of the abstraction, based on the continuity properties of the stochastic kernel of the process and of its jump rate function. We illustrate the approach on a case study of thermostatically controlled loads.

[1]  D. Sherrington.  Stochastic Processes in Physics and Chemistry , 1983 .

[2]  Christel Baier,et al.  Model Checking Continuous-Time Markov Chains by Transient Analysis , 2000, CAV.

[3]  John Lygeros,et al.  Probabilistic reachability and safety for controlled discrete time stochastic hybrid systems , 2008, Autom..

[4]  Alessandro Abate,et al.  Adaptive and Sequential Gridding Procedures for the Abstraction and Verification of Stochastic Processes , 2013, SIAM J. Appl. Dyn. Syst..

[5]  Joost-Pieter Katoen,et al.  Quantitative Model Checking of Continuous-Time Markov Chains Against Timed Automata Specifications , 2009, 2009 24th Annual IEEE Symposium on Logic In Computer Science.

[6]  Nicola Bruti-Liberati,et al.  Stochastic Differential Equations with Jumps , 2010 .

[7]  Christel Baier,et al.  Model-Checking Algorithms for Continuous-Time Markov Chains , 2002, IEEE Trans. Software Eng..

[8]  Johanna L. Mathieu,et al.  State Estimation and Control of Electric Loads to Manage Real-Time Energy Imbalance , 2013 .

[9]  Y. Ziai.  Statistical models of claim amount distributions in general insurance , 1979 .

[10]  Alessandro Abate,et al.  Adaptive Gridding for Abstraction and Verification of Stochastic Hybrid Systems , 2011, 2011 Eighth International Conference on Quantitative Evaluation of SysTems.

[11]  Alessandro Abate,et al.  Precise Approximations of the Probability Distribution of a Markov Process in Time: An Application to Probabilistic Invariance , 2014, TACAS.

[12]  M. Micheli.  Random Sampling of a Continuous-time Stochastic Dynamical System: Analysis, State Estimation, and Applications , 2001 .

[13]  Alessandro Abate,et al.  Higher-Order Approximations for Verification of Stochastic Hybrid Systems , 2012, ATVA.

[14]  O. Kallenberg.  Foundations of Modern Probability , 2021, Probability Theory and Stochastic Modelling.

[15]  Prakash Panangaden,et al.  Labelled Markov Processes , 2009 .

[16]  Dimitri P. Bertsekas,et al.  Stochastic optimal control : the discrete time case , 2007 .

[17]  Mark H. Davis.  Markov Models and Optimization , 1995 .

[18]  Joost-Pieter Katoen,et al.  A Markov reward model checker , 2005, Second International Conference on the Quantitative Evaluation of Systems (QEST'05).

[19]  Alessandro Abate,et al.  Characterization and computation of infinite-horizon specifications over Markov processes , 2012, Theor. Comput. Sci..

[20]  Andrew Papanicolaou.  Numerical Solution of Stochastic Differential Equations with Jumps in Finance , 2013 .

[21]  Andrew Hinton,et al.  PRISM: A Tool for Automatic Verification of Probabilistic Systems , 2006, TACAS.

[22]  Duncan S. Callaway.  Tapping the energy storage potential in electric loads to deliver load following and regulation, with application to wind energy , 2009 .

[23]  Christel Baier,et al.  Principles of model checking , 2008 .

[24]  Alessandro Abate,et al.  Aggregation and Control of Populations of Thermostatically Controlled Loads by Formal Abstractions , 2015, IEEE Transactions on Control Systems Technology.

[25]  R. Bass,et al.  Review: P. Billingsley, Convergence of probability measures , 1971 .

[26]  Alessandro Abate,et al.  On infinite-horizon probabilistic properties and stochastic bisimulation functions , 2011, IEEE Conference on Decision and Control and European Control Conference.

[27]  Joost-Pieter Katoen,et al.  Approximate Model Checking of Stochastic Hybrid Systems , 2010, Eur. J. Control.

[28]  A. Shiryaev,et al.  Limit Theorems for Stochastic Processes , 1987 .

[29]  Christel Baier,et al.  Approximate Symbolic Model Checking of Continuous-Time Markov Chains , 1999, CONCUR.

[30]  Alessandro Abate,et al.  FAUST 2 : Formal Abstractions of Uncountable-STate STochastic Processes , 2014, TACAS.

[31]  Taolue Chen,et al.  Time-Bounded Verification of CTMCs against Real-Time Specifications , 2011, FORMATS.

[32]  Yacine Ait-Sahalia,et al.  The Effects of Random and Discrete Sampling When Estimating Continuous-Time Diffusions , 2002 .

[33]  Alessandro Abate,et al.  Quantitative Approximation of the Probability Distribution of a Markov Process by Formal Abstractions , 2015, Log. Methods Comput. Sci..

[34]  I. I. Gikhman,et al.  The Theory of Stochastic Processes II , 1975 .

[35]  Robert K. Brayton,et al.  Verifying Continuous Time Markov Chains , 1996, CAV.

[36]  L. Allen.  An introduction to stochastic processes with applications to biology , 2003 .

[37]  Christel Baier,et al.  Principles of Model Checking (Representation and Mind Series) , 2008 .