Safety Embedded Differential Dynamic Programming using Discrete Barrier States

Certified safe control is a growing challenge in robotics, especially when performance and safety objectives must be concurrently achieved. In this work, we extend the barrier state (BaS) concept, recently proposed for safe stabilization of continuous time systems, to safety embedded trajectory optimization for discrete time systems using discrete barrier states (DBaS). The constructed DBaS is embedded into the discrete model of the safety-critical system integrating safety objectives into the system’s dynamics and performance objectives. Thereby, the control policy is directly supplied by safety-critical information through the barrier state. This allows us to employ the DBaS with differential dynamic programming (DDP) to plan and execute safe optimal trajectories. The proposed algorithm is leveraged on various safety-critical control and planning problems including a differential wheeled robot safe navigation in randomized and complex environments and on a quadrotor to safely perform reaching and tracking tasks. The DBaS-based DDP (DBaS-DDP) is shown to consistently outperform penalty methods commonly used to approximate constrained DDP problems as well as CBFbased safety filters.

[1]  Aaron D. Ames,et al.  Adaptive Safety with Control Barrier Functions , 2019, 2020 American Control Conference (ACC).

[2]  Cheng Qian,et al.  Learning Barrier Functions with Memory for Robust Safe Navigation , 2020, ArXiv.

[3]  Scott Kuindersma,et al.  Constrained unscented dynamic programming , 2017, 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[4]  Li Wang,et al.  Multi-objective compositions for collision-free connectivity maintenance in teams of mobile robots , 2016, 2016 IEEE 55th Conference on Decision and Control (CDC).

[5]  Evangelos A. Theodorou,et al.  Constrained Differential Dynamic Programming Revisited , 2020, 2021 IEEE International Conference on Robotics and Automation (ICRA).

[6]  Li Wang,et al.  Safe Learning of Quadrotor Dynamics Using Barrier Certificates , 2017, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[7]  S. Prajna Barrier certificates for nonlinear model validation , 2003, 42nd IEEE International Conference on Decision and Control (IEEE Cat. No.03CH37475).

[8]  S. Yakowitz,et al.  Constrained differential dynamic programming and its application to multireservoir control , 1979 .

[9]  Joel W. Burdick,et al.  Safe Policy Synthesis in Multi-Agent POMDPs via Discrete-Time Barrier Functions , 2019, 2019 IEEE 58th Conference on Decision and Control (CDC).

[10]  Calin Belta,et al.  Control Barrier Functions for Systems with High Relative Degree , 2019, 2019 IEEE 58th Conference on Decision and Control (CDC).

[11]  David Harris Jacobson,et al.  Differential dynamic programming methods for determining optimal control of non-linear systems , 1967 .

[12]  Koushil Sreenath,et al.  Reinforcement Learning for Safety-Critical Control under Model Uncertainty, using Control Lyapunov Functions and Control Barrier Functions , 2020, Robotics: Science and Systems.

[13]  Franco Blanchini,et al.  Set invariance in control , 1999, Autom..

[14]  Yuval Tassa,et al.  Control-limited differential dynamic programming , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[15]  Aharon Ben-Tal,et al.  Lectures on modern convex optimization , 1987 .

[16]  Koushil Sreenath,et al.  Discrete Control Barrier Functions for Safety-Critical Control of Discrete Systems with Application to Bipedal Robot Navigation , 2017, Robotics: Science and Systems.

[17]  Evangelos A. Theodorou,et al.  Safe Optimal Control Using Stochastic Barrier Functions and Deep Forward-Backward SDEs , 2020, ArXiv.

[18]  C. Karen Liu,et al.  Differential dynamic programming with nonlinear constraints , 2017, 2017 IEEE International Conference on Robotics and Automation (ICRA).

[19]  Christos G. Cassandras,et al.  Feasibility-Guided Learning for Constrained Optimal Control Problems , 2020, 2020 59th IEEE Conference on Decision and Control (CDC).

[20]  Adrian Wills,et al.  Barrier function based model predictive control , 2004, Autom..

[21]  Paulo Tabuada,et al.  Control barrier function based quadratic programs with application to adaptive cruise control , 2014, 53rd IEEE Conference on Decision and Control.

[22]  Koushil Sreenath,et al.  Exponential Control Barrier Functions for enforcing high relative-degree safety-critical constraints , 2016, 2016 American Control Conference (ACC).

[23]  Frank Allgöwer,et al.  CONSTRUCTIVE SAFETY USING CONTROL BARRIER FUNCTIONS , 2007 .

[24]  Nader Sadegh,et al.  Safety Embedded Control of Nonlinear Systems via Barrier States , 2021, IEEE Control Systems Letters.

[25]  Stephen Prajna Barrier certificates for nonlinear model validation , 2006, Autom..

[26]  Ali Jadbabaie,et al.  Safety Verification of Hybrid Systems Using Barrier Certificates , 2004, HSCC.

[27]  Chris Manzie,et al.  Interior Point Differential Dynamic Programming , 2020, IEEE Transactions on Control Systems Technology.

[28]  D. Mayne A Second-order Gradient Method for Determining Optimal Trajectories of Non-linear Discrete-time Systems , 1966 .

[29]  Zachary Manchester,et al.  ALTRO: A Fast Solver for Constrained Trajectory Optimization , 2019, 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[30]  Francesco Sabatino,et al.  Quadrotor control: modeling, nonlinearcontrol design, and simulation , 2015 .

[31]  David Q. Mayne,et al.  Differential dynamic programming , 1972, The Mathematical Gazette.

[32]  Bayu Jayawardhana,et al.  Uniting Control Lyapunov and Control Barrier Functions , 2014, 53rd IEEE Conference on Decision and Control.

[33]  Paulo Tabuada,et al.  Control Barrier Function Based Quadratic Programs for Safety Critical Systems , 2016, IEEE Transactions on Automatic Control.