Discrete time stochastic hybrid dynamical games: Verification & controller synthesis

This paper presents a framework for analyzing probabilistic safety and reachability problems for discrete time stochastic hybrid systems in scenarios where system dynamics are affected by rational competing agents. In particular, we consider a zero-sum game formulation of the probabilistic reach-avoid problem, in which the control objective is to maximize the probability of reaching a desired subset of the hybrid state space, while avoiding an unsafe set, subject to the worst-case behavior of a rational adversary. Theoretical results are provided on a dynamic programming algorithm for computing the maximal reach-avoid probability under the worst-case adversary strategy, as well as the existence of a max-min control policy which achieves this probability. The modeling framework and computational algorithm are demonstrated using an example derived from a robust motion planning application.

[1]  Jianghai Hu,et al.  Application of Reachability Analysis for Stochastic Hybrid Systems to Aircraft Conflict Prediction , 2008, IEEE Transactions on Automatic Control.

[2]  Claire J. Tomlin,et al.  Reachability-based synthesis of feedback policies for motion planning under bounded disturbances , 2011, 2011 IEEE International Conference on Robotics and Automation.

[3]  E. Feron,et al.  Robust hybrid control for autonomous vehicle motion planning , 2000, Proceedings of the 39th IEEE Conference on Decision and Control (Cat. No.00CH37187).

[4]  S. Shankar Sastry,et al.  Safe and Secure Networked Control Systems under Denial-of-Service Attacks , 2009, HSCC.

[5]  John Lygeros,et al.  A stochastic reach-avoid problem with random obstacles , 2011, HSCC '11.

[6]  John Lygeros,et al.  Verification of discrete time stochastic hybrid systems: A stochastic reach-avoid decision problem , 2010, Autom..

[7]  Aaron D. Ames,et al.  Three-Dimensional Kneed Bipedal Walking: A Hybrid Geometric Approach , 2009, HSCC.

[8]  C. Tomlin,et al.  Symbolic reachable set computation of piecewise affine hybrid automata and its application to biological modelling: Delta-Notch protein signalling. , 2004, Systems biology.

[9]  Andrzej S. Nowak,et al.  On measurable minimax selectors , 2010 .

[10]  William D. Sudderth,et al.  Finitely additive stochastic games with Borel measurable payoffs , 1998, Int. J. Game Theory.

[11]  Bohn Stafleu van Loghum,et al.  Online … , 2002, LOG IN.

[12]  Ulrich Rieder,et al.  Non-Cooperative Dynamic Games with General Utility Functions , 1991 .

[13]  L. Brown,et al.  Measurable Selections of Extrema , 1973 .

[14]  S. Sastry,et al.  Towars a Theory of Stochastic Hybrid Systems , 2000, HSCC.

[15]  A. Nowak Universally Measurable Strategies in Zero-Sum Stochastic Games , 1985 .

[16]  Joost-Pieter Katoen,et al.  Approximate Model Checking of Stochastic Hybrid Systems , 2010, Eur. J. Control.

[17]  Alessandro Abate,et al.  Adaptive Gridding for Abstraction and Verification of Stochastic Hybrid Systems , 2011, 2011 Eighth International Conference on Quantitative Evaluation of SysTems.

[18]  Ashish Tiwari,et al.  Symbolic Systems Biology: Hybrid Modeling and Analysis of Biological Networks , 2004, HSCC.

[19]  Manuela L. Bujorianu,et al.  Extended Stochastic Hybrid Systems and Their Reachability Problem , 2004, HSCC.

[20]  P. Kumar,et al.  Existence of Value and Randomized Strategies in Zero-Sum Discrete-Time Stochastic Dynamic Games , 1981 .

[21]  John Lygeros,et al.  A Dynamic Game Framework for Verification and Control of Stochastic Hybrid Systems , 2011 .

[22]  D. D. Sworder,et al.  Minimax Control of Discrete Time Stochastic Systems , 1964 .

[23]  George J. Pappas,et al.  Hybrid control in air traffic management systems , 1995, Proceedings of 1995 34th IEEE Conference on Decision and Control.

[24]  George J. Pappas,et al.  Stochastic safety verification using barrier certificates , 2004, 2004 43rd IEEE Conference on Decision and Control (CDC) (IEEE Cat. No.04CH37601).

[25]  João Pedro Hespanha,et al.  Stochastic Hybrid Systems: Application to Communication Networks , 2004, HSCC.

[26]  S. Shankar Sastry,et al.  Conflict resolution for air traffic management: a study in multiagent hybrid systems , 1998, IEEE Trans. Autom. Control..

[27]  Claire J. Tomlin,et al.  Quadrotor Helicopter Flight Dynamics and Control: Theory and Experiment , 2007 .

[28]  John Lygeros,et al.  Probabilistic reachability and safety for controlled discrete time stochastic hybrid systems , 2008, Autom..

[29]  George J. Pappas,et al.  Hybrid Control in Air Traffic Management Systems 1 , 1996 .

[30]  Dimitri P. Bertsekas,et al.  Stochastic optimal control : the discrete time case , 2007 .

[31]  Claire J. Tomlin,et al.  Applications of hybrid reachability analysis to robotic aerial vehicles , 2011, Int. J. Robotics Res..

[32]  J. Lygeros,et al.  Probabilistic reachability and safe sets computation for discrete time stochastic hybrid systems , 2006, Proceedings of the 45th IEEE Conference on Decision and Control.

[33]  John Lygeros,et al.  Towars a Theory of Stochastic Hybrid Systems , 2000, HSCC.

[34]  T. Başar,et al.  Dynamic Noncooperative Game Theory , 1982 .

[35]  Xenofon D. Koutsoukos,et al.  Computational Methods for Reachability Analysis of Stochastic Hybrid Systems , 2006, HSCC.

[36]  Alberto Olivares,et al.  Engineering Notes Hybrid Optimal Control Approach to Commercial Aircraft Trajectory Planning , 2010 .