Probabilistic reachability and safety for controlled discrete time stochastic hybrid systems

In this work, probabilistic reachability over a finite horizon is investigated for a class of discrete time stochastic hybrid systems with control inputs. A suitable embedding of the reachability problem in a stochastic control framework reveals that it is amenable to two complementary interpretations, leading to dual algorithms for reachability computations. In particular, the set of initial conditions providing a certain probabilistic guarantee that the system will keep evolving within a desired 'safe' region of the state space is characterized in terms of a value function, and 'maximally safe' Markov policies are determined via dynamic programming. These results are of interest not only for safety analysis and design, but also for solving those regulation and stabilization problems that can be reinterpreted as safety problems. The temperature regulation problem presented in the paper as a case study is one such case.

[1]  S. Shankar Sastry,et al.  Aircraft conflict prediction in the presence of a spatially correlated wind field , 2005, IEEE Transactions on Intelligent Transportation Systems.

[2]  Alexander B. Kurzhanski,et al.  REACHABILITY ANALYSIS UNDER CONTROL-DEPENDENT STOCHASTIC NOISE , 2005 .

[3]  Pravin Varaiya,et al.  Stochastic Systems: Estimation, Identification, and Adaptive Control , 1986 .

[4]  George J. Pappas,et al.  Discrete abstractions of hybrid systems , 2000, Proceedings of the IEEE.

[5]  Joost P. Katoen,et al.  Stochastic model checking , 2006 .

[6]  Antoine Girard,et al.  Approximate simulation Relations for Hybrid Systems , 2006, ADHS.

[7]  Ansgar Fehnker,et al.  Benchmarks for Hybrid Systems Verification , 2004, HSCC.

[8]  Alexandre M. Bayen,et al.  Computational techniques for the verification of hybrid systems , 2003, Proc. IEEE.

[9]  J. Lygeros,et al.  General stochastic hybrid systems: modelling and optimal control , 2004, 2004 43rd IEEE Conference on Decision and Control (CDC) (IEEE Cat. No.04CH37601).

[10]  Dimitri P. Bertsekas,et al.  Stochastic optimal control : the discrete time case , 2007 .

[11]  Antoine Girard,et al.  Reachability of Uncertain Linear Systems Using Zonotopes , 2005, HSCC.

[12]  Manuela L. Bujorianu,et al.  Extended Stochastic Hybrid Systems and Their Reachability Problem , 2004, HSCC.

[13]  Olivier Bournez,et al.  Approximate Reachability Analysis of Piecewise-Linear Dynamical Systems , 2000, HSCC.

[14]  M. K. Ghosh,et al.  Ergodic Control of Switching Diffusions , 1997 .

[15]  John Lygeros,et al.  Controllers for reachability specifications for hybrid systems , 1999, Autom..

[16]  John Lygeros,et al.  On reachability and minimum cost optimal control , 2004, Autom..

[17]  R. Malhamé,et al.  Electric load model synthesis by diffusion approximation of a high-order hybrid-state stochastic system , 1985 .

[18]  Martin L. Puterman,et al.  Markov Decision Processes: Discrete Stochastic Dynamic Programming , 1994 .

[19]  John Lygeros,et al.  A probabilistic approach to aircraft conflict detection , 2000, IEEE Trans. Intell. Transp. Syst..

[20]  John Lygeros,et al.  Synthesizing Controllers for Nonlinear Hybrid Systems , 1998, HSCC.

[21]  J. Lygeros,et al.  Probabilistic reachability and safe sets computation for discrete time stochastic hybrid systems , 2006, Proceedings of the 45th IEEE Conference on Decision and Control.

[22]  Ádám M. Halász,et al.  Understanding the Bacterial Stringent Response Using Reachability Analysis of Hybrid Systems , 2004, HSCC.

[23]  John Lygeros,et al.  Towars a Theory of Stochastic Hybrid Systems , 2000, HSCC.

[24]  H. Kushner.  Numerical Methods for Stochastic Control Problems in Continuous Time , 2000 .

[25]  Mark H. Davis.  Markov Models and Optimization , 1995 .

[26]  John Lygeros,et al.  Stochastic reachability for discrete time systems: an application to aircraft collision avoidance , 2003, 42nd IEEE International Conference on Decision and Control (IEEE Cat. No.03CH37475).

[27]  Ian M. Mitchell,et al.  A Toolbox of Hamilton-Jacobi Solvers for Analysis of Nondeterministic Continuous and Hybrid Systems , 2005, HSCC.

[28]  Jianghai Hu,et al.  A stochastic approximation method for reachability computations , 2006 .

[29]  Maria Prandini,et al.  Stochastic Reachability: Theory and Numerical Approximation , 2006 .

[30]  D. Bertsekas.  Infinite time reachability of state-space regions by using feedback control , 1972 .

[31]  Alexandre M. Bayen,et al.  Computational Techniques for the Verification and Control of Hybrid Systems , 2005 .

[32]  Pravin Varaiya,et al.  On Reachability Under Uncertainty , 2002, SIAM J. Control. Optim..

[33]  George J. Pappas,et al.  A Framework for Worst-Case and Stochastic Safety Verification Using Barrier Certificates , 2007, IEEE Transactions on Automatic Control.

[34]  Ian M. Mitchell,et al.  Level Set Methods for Computation in Hybrid Systems , 2000, HSCC.

[35]  Tiziano Villa,et al.  Maximal Safe Set Computation for Idle Speed Control of an Automotive Engine , 2000, HSCC.