Computational Methods for Reachability Analysis of Stochastic Hybrid Systems

Stochastic hybrid system models can be used to analyze and design complex embedded systems that operate in the presence of uncertainty and variability. Verification of reachability properties for such systems is a critical problem. Developing algorithms for reachability analysis is challenging because of the interaction between the discrete and continuous stochastic dynamics. In this paper, we propose a probabilistic method for reachability analysis based on discrete approximations. The contribution of the paper is twofold. First, we show that reachability can be characterized as a viscosity solution of a system of coupled Hamilton-Jacobi-Bellman equations. Second, we present a numerical method for computing the solution based on discrete approximations and we show that this solution converges to the one for the original system as the discretization becomes finer. Finally, we illustrate the approach with a navigation benchmark that has been proposed for hybrid system verification.

[1]  Alexandre M. Bayen,et al.  A time-dependent Hamilton-Jacobi formulation of reachable sets for continuous dynamic games , 2005, IEEE Transactions on Automatic Control.

[2]  Mark H. A. Davis,et al.  Piecewise-Deterministic Processes and Viscosity Solutions , 1999 .

[3]  Dimitri P. Bertsekas,et al.  Stochastic optimal control : the discrete time case , 2007 .

[4]  Ian M. Mitchell,et al.  A Toolbox of Hamilton-Jacobi Solvers for Analysis of Nondeterministic Continuous and Hybrid Systems , 2005, HSCC.

[5]  X. Koutsoukos.  Optimal Control of Stochastic Hybrid Systems Based on Locally Consistent Markov Decision Processes , 2005, Proceedings of the 2005 IEEE International Symposium on, Mediterrean Conference on Control and Automation Intelligent Control, 2005..

[6]  J. Lygeros,et al.  General stochastic hybrid systems: modelling and optimal control , 2004, 2004 43rd IEEE Conference on Decision and Control (CDC) (IEEE Cat. No.04CH37601).

[7]  Ansgar Fehnker,et al.  Benchmarks for Hybrid Systems Verification , 2004, HSCC.

[8]  Rajeev Alur,et al.  Structured Modeling of Concurrent Stochastic Hybrid Systems , 2004, FORMATS/FTRTFT.

[9]  Arjan van der Schaft,et al.  Communicating Piecewise Deterministic Markov Processes , 2003, ADHS.

[10]  João Pedro Hespanha.  Polynomial Stochastic Hybrid Systems , 2005, HSCC.

[11]  John Lygeros,et al.  On reachability and minimum cost optimal control , 2004, Autom..

[12]  S. Shankar Sastry,et al.  Probabilistic safety analysis in three dimensional aircraft flight , 2003, 42nd IEEE International Conference on Decision and Control (IEEE Cat. No.03CH37475).

[13]  John Lygeros,et al.  Reachability Questions in Piecewise Deterministic Markov Processes , 2003, HSCC.

[14]  John Lygeros,et al.  Bisimulation for General Stochastic Hybrid Systems , 2005, HSCC.

[15]  Luca de Alfaro,et al.  Computing Minimum and Maximum Reachability Times in Probabilistic Systems , 1999, CONCUR.

[16]  Mihalis Yannakakis,et al.  The complexity of probabilistic verification , 1995, JACM.

[17]  João Pedro Hespanha,et al.  Stochastic Hybrid Systems: Application to Communication Networks , 2004, HSCC.

[18]  S. Sastry,et al.  Towars a Theory of Stochastic Hybrid Systems , 2000, HSCC.

[19]  W. Fleming,et al.  Controlled Markov processes and viscosity solutions , 1992 .

[20]  Martin L. Puterman,et al.  Markov Decision Processes: Discrete Stochastic Dynamic Programming , 1994 .

[21]  John Lygeros,et al.  Towars a Theory of Stochastic Hybrid Systems , 2000, HSCC.

[22]  H. Kushner.  Numerical Methods for Stochastic Control Problems in Continuous Time , 2000 .

[23]  Mark H. Davis.  Markov Models and Optimization , 1995 .