A Classification-based Approach for Approximate Reachability

Hamilton-Jacobi (HJ) reachability analysis has been developed over the past decades into a widely-applicable tool for determining goal satisfaction and safety verification in nonlinear systems. While HJ reachability can be formulated very generally, computational complexity can be a serious impediment for many systems of practical interest. Much prior work has been devoted to computing approximate solutions to large reachability problems, yet many of these methods may only apply to very restrictive problem classes, do not generate controllers, and/or can be extremely conservative. In this paper, we present a new method for approximating the optimal controller of the HJ reachability problem for control-affine systems. While also a specific problem class, many dynamical systems of interest are, or can be well approximated, by control-affine models. We explicitly avoid storing a representation of the reachability value function, and instead learn a controller as a sequence of simple binary classifiers. We compare our approach to existing grid-based methodologies in HJ reachability and demonstrate its utility on several examples, including a physical quadrotor navigation task.

[1]  Mo Chen,et al.  FaSTrack: A modular framework for fast and guaranteed safe motion planning , 2017, 2017 IEEE 56th Annual Conference on Decision and Control (CDC).

[2]  John Lygeros,et al.  Hybrid Systems: Modeling, Analysis and Control , 2008 .

[3]  P. Souganidis,et al.  Differential Games and Representation Formulas for Solutions of Hamilton-Jacobi-Isaacs Equations. , 1983 .

[4]  Mo Chen,et al.  Robust Sequential Trajectory Planning Under Disturbances and Adversarial Intruder , 2019, IEEE Transactions on Control Systems Technology.

[5]  Sergey Levine,et al.  Neural Network Dynamics for Model-Based Deep Reinforcement Learning with Model-Free Fine-Tuning , 2017, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[6]  Russ Tedrake,et al.  Convex optimization of nonlinear feedback controllers via occupation measures , 2013, Int. J. Robotics Res..

[7]  Mo Chen,et al.  Safe platooning of unmanned aerial vehicles via reachability , 2015, 2015 54th IEEE Conference on Decision and Control (CDC).

[8]  Mo Chen,et al.  Fast reachable set approximations via state decoupling disturbances , 2016, 2016 IEEE 55th Conference on Decision and Control (CDC).

[9]  Ian M. Mitchell,et al.  Lagrangian methods for approximating the viability kernel in high-dimensional systems , 2013, Autom..

[10]  Mo Chen,et al.  Using Neural Networks to Compute Approximate and Guaranteed Feasible Hamilton-Jacobi-Bellman PDE Solutions , 2016, 1611.03158.

[11]  Claire J. Tomlin,et al.  Learning quadrotor dynamics using neural network for flight control , 2016, 2016 IEEE 55th Conference on Decision and Control (CDC).

[12]  Pieter Abbeel,et al.  Reverse Curriculum Generation for Reinforcement Learning , 2017, CoRL.

[13]  Mo Chen,et al.  Hamilton-Jacobi reachability: A brief overview and recent advances , 2017, 2017 IEEE 56th Annual Conference on Decision and Control (CDC).

[14]  Antoine Girard,et al.  SpaceEx: Scalable Verification of Hybrid Systems , 2011, CAV.

[15]  Ian M. Mitchell,et al.  Computing the viability kernel using maximal reachable sets , 2012, HSCC '12.

[16]  Cong Wang,et al.  Dynamic Learning From Adaptive Neural Network Control of a Class of Nonaffine Nonlinear Systems , 2014, IEEE Transactions on Neural Networks and Learning Systems.

[17]  Ian M. Mitchell.  The Flexible, Extensible and Efficient Toolbox of Level Set Methods , 2008, J. Sci. Comput..

[18]  Mo Chen,et al.  Exact and efficient Hamilton-Jacobi guaranteed safety analysis via system decomposition , 2017, 2017 IEEE International Conference on Robotics and Automation (ICRA).

[19]  Mo Chen,et al.  Reach-avoid problems with time-varying dynamics, targets and constraints , 2014, HSCC.

[20]  Pravin Varaiya,et al.  Ellipsoidal Techniques for Reachability Analysis , 2000, HSCC.

[21]  Carla Piazza,et al.  Parallelotope Bundles for Polynomial Reachability , 2016, HSCC.

[22]  P. Varaiya,et al.  Ellipsoidal techniques for reachability analysis: internal approximation , 2000 .

[23]  Ian M. Mitchell,et al.  Integrating Projections , 1998, HSCC.

[24]  Petter Nilsson,et al.  Synthesis of separable controlled invariant sets for modular local control design , 2015, 2016 American Control Conference (ACC).

[25]  Sergey Levine,et al.  End-to-End Training of Deep Visuomotor Policies , 2015, J. Mach. Learn. Res..

[26]  Claire J. Tomlin,et al.  A differential game approach to planning in adversarial scenarios: A case study on capture-the-flag , 2011, 2011 IEEE International Conference on Robotics and Automation.

[27]  P. Varaiya.  On the Existence of Solutions to a Differential Game , 1967 .

[28]  Pravin Varaiya,et al.  On Ellipsoidal Techniques for Reachability Analysis. Part II: Internal Approximations Box-valued Constraints , 2002, Optim. Methods Softw..

[29]  Hasnaa Zidani,et al.  Reachability and Minimal Times for State Constrained Nonlinear Problems without Any Controllability Assumption , 2010, SIAM J. Control. Optim..

[30]  Ian M. Mitchell,et al.  A Toolbox of Level Set Methods , 2005 .

[31]  Sergey Levine,et al.  PLATO: Policy learning using adaptive trajectory optimization , 2016, 2017 IEEE International Conference on Robotics and Automation (ICRA).

[32]  Mo Chen,et al.  Decomposition of Reachable Sets and Tubes for a Class of Nonlinear Systems , 2016, IEEE Transactions on Automatic Control.

[33]  Matthias Althoff,et al.  An Introduction to CORA 2015 , 2015, ARCH@CPSWeek.

[34]  Huaguang Zhang,et al.  Neural-Network-Based Near-Optimal Control for a Class of Discrete-Time Affine Nonlinear Systems With Control Constraints , 2009, IEEE Transactions on Neural Networks.

[35]  E. Barron.  Differential games maximum cost , 1990 .

[36]  Igor Skrjanc,et al.  Model-based Predictive Control of Hybrid Systems: A Probabilistic Neural-network Approach to Real-time Control , 2008, J. Intell. Robotic Syst..

[37]  J. Lygeros,et al.  Neural approximation of PDE solutions: An application to reachability computations , 2006, Proceedings of the 45th IEEE Conference on Decision and Control.

[38]  Ronald Fedkiw,et al.  Level set methods and dynamic implicit surfaces , 2002, Applied mathematical sciences.

[39]  Alexandre M. Bayen,et al.  Aircraft Autolander Safety Analysis Through Optimal Control-Based Reach Set Computation , 2007 .

[40]  S. Shankar Sastry,et al.  Reachability calculations for automated aerial refueling , 2008, 2008 47th IEEE Conference on Decision and Control.

[41]  J A Sethian,et al.  A fast marching level set method for monotonically advancing fronts. , 1996, Proceedings of the National Academy of Sciences of the United States of America.

[42]  Daniel Liberzon,et al.  Calculus of Variations and Optimal Control Theory: A Concise Introduction , 2012 .

[43]  Alexandre M. Bayen,et al.  A time-dependent Hamilton-Jacobi formulation of reachable sets for continuous dynamic games , 2005, IEEE Transactions on Automatic Control.

[44]  Xin Chen,et al.  Flow*: An Analyzer for Non-linear Hybrid Systems , 2013, CAV.