On infinite-horizon probabilistic properties and stochastic bisimulation functions

This work investigates infinite-horizon properties over discrete-time stochastic models with continuous state spaces. The focus is on understanding how the structural features of a model (e.g., the presence of absorbing sets) affect the values of these properties and relate to their uniqueness. Furthermore, we argue that the investigation of these features can lead to approximation bounds for the value of such properties, as well as to improvements on their computation. The article employs the presented results to find a stochastic bisimulation function of two processes.

[1]  J. Lygeros,et al.  Probabilistic reachability and safe sets computation for discrete time stochastic hybrid systems , 2006, Proceedings of the 45th IEEE Conference on Decision and Control.

[2]  H. Kushner.  Stochastic Stability and Control , 2012 .

[3]  Richard L. Tweedie,et al.  Markov Chains and Stochastic Stability , 1993, Communications and Control Engineering Series.

[4]  Martin Fränzle,et al.  Engineering constraint solvers for automatic analysis of probabilistic hybrid automata , 2010, J. Log. Algebraic Methods Program..

[5]  John Lygeros,et al.  New insights on stochastic reachability , 2007, 2007 46th IEEE Conference on Decision and Control.

[6]  R. Durrett.  Probability: Theory and Examples , 1993 .

[7]  Alʹbert Nikolaevich Shiri︠a︡ev,et al.  Optimal Stopping and Free-Boundary Problems , 2006 .

[8]  Joost-Pieter Katoen,et al.  Quantitative automata model checking of autonomous stochastic hybrid systems , 2011, HSCC '11.

[9]  B. Rozovskii,et al.  Optimal Stopping Rules , 1978 .

[10]  John Lygeros,et al.  Probabilistic reachability and safety for controlled discrete time stochastic hybrid systems , 2008, Autom..

[11]  O. Hernández-Lerma,et al.  Further topics on discrete-time Markov control processes , 1999 .

[12]  John Lygeros,et al.  On the connections between PCTL and dynamic programming , 2009, HSCC '10.

[13]  Joost-Pieter Katoen,et al.  Approximate Model Checking of Stochastic Hybrid Systems , 2010, Eur. J. Control.

[14]  Albert N. Shiryaev,et al.  Optimal Stopping Rules , 1980, International Encyclopedia of Statistical Science.

[15]  George J. Pappas,et al.  Approximations of Stochastic Hybrid Systems , 2009, IEEE Transactions on Automatic Control.