A stochastic reach-avoid problem with random obstacles

We present a dynamic programming based solution to a stochastic reachability problem for a controlled discrete-time stochastic hybrid system. A sum-multiplicative cost function is introduced along with a corresponding dynamic recursion which quantifies the probability of hitting a target set at some point during a finite time horizon, while avoiding an obstacle set during each time step preceding the target hitting time. In contrast with earlier works which consider the reach and avoid sets as both deterministic and time invariant, we consider the avoid set to be both time-varying and probabilistic. Optimal reach-avoid control policies are derived as the solution to an optimal control problem via dynamic programming. A computational example motivated by aircraft motion planning is provided.

[1]  G. Matheron.  Random Sets and Integral Geometry , 1976 .

[2]  W. Rudin.  Real and complex analysis, 3rd ed. , 1987 .

[3]  N. Cressie,et al.  Random set theory and problems of modeling , 1987 .

[4]  Jean-Pierre Aubin,et al.  Viability theory , 1991 .

[5]  Mark H. Davis.  Markov Models and Optimization , 1995 .

[6]  M. K. Ghosh,et al.  Ergodic Control of Switching Diffusions , 1997 .

[7]  Steven A. Amburn,et al.  VIL Density as a Hail Indicator , 1997 .

[8]  D. Stoyan.  Random Sets: Models and Statistics , 1998 .

[9]  S. Shankar Sastry,et al.  Conflict resolution for air traffic management: a study in multiagent hybrid systems , 1998, IEEE Trans. Autom. Control..

[10]  Thomas A. Henzinger,et al.  Hybrid Systems: Computation and Control , 1998, Lecture Notes in Computer Science.

[11]  S. Sastry,et al.  Towars a Theory of Stochastic Hybrid Systems , 2000, HSCC.

[12]  Ian M. Mitchell,et al.  Level Set Methods for Computation in Hybrid Systems , 2000, HSCC.

[13]  J. Lygeros,et al.  A game theoretic approach to controller design for hybrid systems , 2000, Proceedings of the IEEE.

[14]  John Lygeros,et al.  Towars a Theory of Stochastic Hybrid Systems , 2000, HSCC.

[15]  Mark R. Greenstreet,et al.  Hybrid Systems: Computation and Control , 2002, Lecture Notes in Computer Science.

[16]  J. Evans,et al.  THE CORRIDOR INTEGRA TED WEATHER SYSTEM (CIWS) , 2002 .

[17]  B. Boudevillain,et al.  Assessment of Vertically Integrated Liquid (VIL) Water Content Radar Measurement , 2003 .

[18]  John Lygeros,et al.  Reachability Questions in Piecewise Deterministic Markov Processes , 2003, HSCC.

[19]  Alexandre M. Bayen,et al.  Computational techniques for the verification of hybrid systems , 2003, Proc. IEEE.

[20]  John Lygeros,et al.  On reachability and minimum cost optimal control , 2004, Autom..

[21]  Sean R Eddy,et al.  What is dynamic programming? , 2004, Nature Biotechnology.

[22]  I. Molchanov.  Theory of Random Sets , 2005 .

[23]  John Lygeros,et al.  Reachability Analysis for Controlled Discrete Time Stochastic Hybrid Systems , 2006, HSCC.

[24]  Joao P. Hespanha,et al.  Hybrid systems : computation and control : 9th International Workshop, HSCC 2006, Santa Barbara, CA, USA, March 29-31, 2006 : proceedings , 2006 .

[25]  John Lygeros,et al.  Computational Approaches to Reachability Analysis of Stochastic Hybrid Systems , 2007, HSCC.

[26]  Dimitri P. Bertsekas,et al.  Stochastic optimal control : the discrete time case , 2007 .

[27]  Yan Gao,et al.  On the Reachability Problem for Uncertain Hybrid Systems , 2007, IEEE Transactions on Automatic Control.

[28]  John Lygeros,et al.  Probabilistic reachability and safety for controlled discrete time stochastic hybrid systems , 2008, Autom..

[29]  Ian M. Mitchell.  The Flexible, Extensible and Efficient Toolbox of Level Set Methods , 2008, J. Sci. Comput..

[30]  John Lygeros,et al.  On the connections between PCTL and dynamic programming , 2009, HSCC '10.

[31]  John Lygeros,et al.  Verification of discrete time stochastic hybrid systems: A stochastic reach-avoid decision problem , 2010, Autom..

[32]  R. Cooke.  Real and Complex Analysis , 2011 .