Characterization and computation of infinite-horizon specifications over Markov processes

This work is devoted to the formal verification of specifications over general discrete-time Markov processes, with an emphasis on infinite-horizon properties. These properties, formulated in a modal logic known as PCTL, can be expressed through value functions defined over the state space of the process. The main goal is to understand how structural features of the model (primarily the presence of absorbing sets) influence the uniqueness of the solutions of corresponding Bellman equations. Furthermore, this contribution shows that the investigation of these structural features leads to new computational techniques to calculate the specifications of interest: the emphasis is to derive approximation techniques with associated explicit convergence rates and formal error bounds.

[1]  P. Moerbeke.  On optimal stopping and free boundary problems , 1973, Advances in Applied Probability.

[2]  B. Rozovskii,et al.  Optimal Stopping Rules , 1978 .

[3]  Andrew Hinton,et al.  PRISM: A Tool for Automatic Verification of Probabilistic Systems , 2006, TACAS.

[4]  R. Cooke.  Real and Complex Analysis , 2011 .

[5]  John Lygeros,et al.  Probabilistic reachability and safety for controlled discrete time stochastic hybrid systems , 2008, Autom..

[6]  W. Rudin.  Real and complex analysis , 1968 .

[7]  Paulo Tabuada,et al.  Verification and Control of Hybrid Systems - A Symbolic Approach , 2009 .

[8]  Martin Leucker,et al.  Don't Know in Probabilistic Systems , 2006, SPIN.

[9]  John Lygeros,et al.  Verification of discrete time stochastic hybrid systems: A stochastic reach-avoid decision problem , 2010, Autom..

[10]  Sean P. Meyn.  Control Techniques for Complex Networks: Workload , 2007 .

[11]  Marta Z. Kwiatkowska,et al.  Automated Verification Techniques for Probabilistic Systems , 2011, SFM.

[12]  O. Kallenberg.  Foundations of Modern Probability , 2021, Probability Theory and Stochastic Modelling.

[13]  E. Nummelin.  General irreducible Markov chains and non-negative operators: List of symbols and notation , 1984 .

[14]  H. Kushner.  Stochastic Stability and Control , 2012 .

[15]  D. Vere-Jones.  Markov Chains , 1972, Nature.

[16]  Alessandro Abate,et al.  Adaptive and Sequential Gridding Procedures for the Abstraction and Verification of Stochastic Processes , 2013, SIAM J. Appl. Dyn. Syst..

[17]  W. Rudin.  Principles of mathematical analysis , 1964 .

[18]  O. Hernández-Lerma,et al.  Discrete-time Markov control processes , 1999 .

[19]  Alessandro Abate,et al.  Probabilistic invariance of mixed deterministic-stochastic dynamical systems , 2012, HSCC '12.

[20]  Christel Baier,et al.  Principles of Model Checking (Representation and Mind Series) , 2008 .

[21]  R. Durrett.  Probability: Theory and Examples , 1993 .

[22]  W. Rudin.  Real and complex analysis, 3rd ed. , 1987 .

[23]  Christel Baier,et al.  Principles of model checking , 2008 .

[24]  Joost-Pieter Katoen,et al.  Quantitative automata model checking of autonomous stochastic hybrid systems , 2011, HSCC '11.

[25]  Joost-Pieter Katoen,et al.  A Markov reward model checker , 2005, Second International Conference on the Quantitative Evaluation of Systems (QEST'05).

[26]  Alessandro Abate,et al.  Regularization of bellman equations for infinite-horizon probabilistic properties , 2012, HSCC '12.

[27]  O. Hernández-Lerma.  Adaptive Markov Control Processes , 1989 .

[28]  Dimitri P. Bertsekas,et al.  Stochastic optimal control : the discrete time case , 2007 .

[29]  Anna Philippou,et al.  Tools and Algorithms for the Construction and Analysis of Systems , 2018, Lecture Notes in Computer Science.

[30]  Michael Huth,et al.  On finite-state approximants for probabilistic computation tree logic , 2005, Theor. Comput. Sci..

[31]  Alessandro Abate,et al.  On infinite-horizon probabilistic properties and stochastic bisimulation functions , 2011, IEEE Conference on Decision and Control and European Control Conference.

[32]  Joost-Pieter Katoen,et al.  Approximate Model Checking of Stochastic Hybrid Systems , 2010, Eur. J. Control.

[33]  R. Tweedie,et al.  Markov Chains with Continuous Components , 1979 .

[34]  John Lygeros,et al.  On the connections between PCTL and dynamic programming , 2009, HSCC '10.

[35]  Albert N. Shiryaev,et al.  Optimal Stopping Rules , 2011, International Encyclopedia of Statistical Science.

[36]  Michael Huth,et al.  Three-Valued Abstractions of Markov Chains: Completeness for a Sizeable Fragment of PCTL , 2009, FCT.

[37]  George J. Pappas.  Bisimilar linear systems , 2003, Autom..

[38]  Radha Jagadeesan,et al.  Metrics for labelled Markov processes , 2004, Theor. Comput. Sci..

[39]  Alessandro Abate,et al.  Adaptive Gridding for Abstraction and Verification of Stochastic Hybrid Systems , 2011, 2011 Eighth International Conference on Quantitative Evaluation of SysTems.

[40]  Richard L. Tweedie,et al.  Markov Chains and Stochastic Stability , 1993, Communications and Control Engineering Series.

[41]  Valérie Issarny,et al.  Formal Methods for Eternal Networked Software Systems , 2011, Lecture Notes in Computer Science.