Reachability Analysis for Controlled Discrete Time Stochastic Hybrid Systems

A model for discrete time stochastic hybrid systems whose evolution can be influenced by some control input is proposed in this paper. With reference to the introduced class of systems, a methodology for probabilistic reachability analysis is developed that is relevant to safety verification. This methodology is based on the interpretation of the safety verification problem as an optimal control problem for a certain controlled Markov process. In particular, this allows to characterize through some optimal cost function the set of initial conditions for the system such that safety is guaranteed with sufficiently high probability. The proposed methodology is applied to the problem of regulating the average temperature in a room by a thermostat controlling a heater.

[1]  R. Malhamé,et al.  Electric load model synthesis by diffusion approximation of a high-order hybrid-state stochastic system , 1985 .

[2]  Martin L. Puterman,et al.  Markov Decision Processes: Discrete Stochastic Dynamic Programming , 1994 .

[3]  Pravin Varaiya,et al.  Stochastic Systems: Estimation, Identification, and Adaptive Control , 1986 .

[4]  John Lygeros,et al.  Reachability Questions in Piecewise Deterministic Markov Processes , 2003, HSCC.

[5]  John Lygeros,et al.  Towars a Theory of Stochastic Hybrid Systems , 2000, HSCC.

[6]  Mark H. Davis.  Markov Models and Optimization , 1995 .

[7]  John Lygeros,et al.  Stochastic reachability for discrete time systems: an application to aircraft collision avoidance , 2003, 42nd IEEE International Conference on Decision and Control (IEEE Cat. No.03CH37475).

[8]  E. Helfand.  Numerical integration of stochastic differential equations , 1979, The Bell System Technical Journal.

[9]  J. Lygeros,et al.  General stochastic hybrid systems: modelling and optimal control , 2004, 2004 43rd IEEE Conference on Decision and Control (CDC) (IEEE Cat. No.04CH37601).

[10]  Dimitri P. Bertsekas,et al.  Stochastic optimal control : the discrete time case , 2007 .

[11]  George J. Pappas,et al.  Stochastic safety verification using barrier certificates , 2004, 2004 43rd IEEE Conference on Decision and Control (CDC) (IEEE Cat. No.04CH37601).

[12]  M. K. Ghosh,et al.  Ergodic Control of Switching Diffusions , 1997 .

[13]  S. Shankar Sastry,et al.  Probabilistic safety analysis in three dimensional aircraft flight , 2003, 42nd IEEE International Conference on Decision and Control (IEEE Cat. No.03CH37475).

[14]  S. Shankar Sastry,et al.  Aircraft conflict prediction in the presence of a spatially correlated wind field , 2005, IEEE Transactions on Intelligent Transportation Systems.

[15]  S. Sastry,et al.  Towars a Theory of Stochastic Hybrid Systems , 2000, HSCC.

[16]  John Lygeros,et al.  Bisimulation for General Stochastic Hybrid Systems , 2005, HSCC.