An approximate dynamic programming approach to probabilistic reachability for stochastic hybrid systems

This paper addresses the computational overhead involved in probabilistic reachability computations for a general class of controlled stochastic hybrid systems. An approximate dynamic programming approach is proposed to mitigate the curse of dimensionality issue arising in the solution to the stochastic optimal control reformulation of the probabilistic reachability problem. An algorithm tailored to this problem is introduced and compared with the standard numerical solution to dynamic programming on a benchmark example.

[1]  John Lygeros,et al.  Computational Approaches to Reachability Analysis of Stochastic Hybrid Systems , 2007, HSCC.

[2]  Maria Prandini,et al.  Stochastic Reachability: Theory and Numerical Approximation , 2006 .

[3]  Xenofon D. Koutsoukos,et al.  Computational Methods for Reachability Analysis of Stochastic Hybrid Systems , 2006, HSCC.

[4]  Manuela L. Bujorianu,et al.  Extended Stochastic Hybrid Systems and Their Reachability Problem , 2004, HSCC.

[5]  J. Lygeros,et al.  Neural approximation of PDE solutions: An application to reachability computations , 2006, Proceedings of the 45th IEEE Conference on Decision and Control.

[6]  Benjamin Van Roy,et al.  On the existence of fixed points for approximate value iteration and temporal-difference learning , 2000 .

[7]  Dimitri P. Bertsekas,et al.  Stochastic optimal control : the discrete time case , 2007 .

[8]  John N. Tsitsiklis,et al.  Neuro-Dynamic Programming , 1996, Encyclopedia of Machine Learning.

[9]  Ansgar Fehnker,et al.  Benchmarks for Hybrid Systems Verification , 2004, HSCC.

[10]  C. Cassandras,et al.  Stochastic hybrid systems , 2006 .

[11]  George J. Pappas,et al.  A Framework for Worst-Case and Stochastic Safety Verification Using Barrier Certificates , 2007, IEEE Transactions on Automatic Control.

[12]  van der Arjan Schaft,et al.  Stochastic Hybrid Systems: Theory and Safety Critical Applications , 2006 .

[13]  J. Lygeros,et al.  A Neural Approximation to Continuous Time Reachability Computations , 2006, Proceedings of the 45th IEEE Conference on Decision and Control.

[14]  John Lygeros,et al.  Probabilistic reachability and safety for controlled discrete time stochastic hybrid systems , 2008, Autom..