Training Neural Network Controllers Using Control Barrier Functions in the Presence of Disturbances

Control Barrier Functions (CBF) have been recently utilized in the design of provably safe feedback control laws for nonlinear systems. These feedback control methods typically compute the next control input by solving an online Quadratic Program (QP). Solving QPs in real-time can be a computationally expensive process for resource-constrained systems. In the presence of disturbances, finding CBF-based safe control inputs can get even more time consuming as finding the worst-case of the disturbance requires solving a nonlinear program in general. In this work, we propose to use imitation learning to learn Neural Network based feedback controllers which will satisfy the CBF constraints. In the process, we also develop a new class of High Order CBF for systems under external disturbances. We demonstrate the framework on a unicycle model subject to external disturbances, e.g., wind or currents.

[1]  Stefano Ermon,et al.  Generative Adversarial Imitation Learning , 2016, NIPS.

[2]  Peter J. Gawthrop,et al.  Neural networks for control systems - A survey , 1992, Autom..

[3]  Vijay Kumar,et al.  Approximating Explicit Model Predictive Control Using Constrained Neural Networks , 2018, 2018 Annual American Control Conference (ACC).

[4]  Georgios Fainekos,et al.  Gray-box adversarial testing for control systems with machine learning components , 2018, HSCC.

[5]  Geoffrey J. Gordon,et al.  A Reduction of Imitation Learning and Structured Prediction to No-Regret Online Learning , 2010, AISTATS.

[6]  Aaron D. Ames,et al.  Input-to-State Safety With Control Barrier Functions , 2018, IEEE Control Systems Letters.

[7]  Monimoy Bujarbaruah,et al.  Near-Optimal Rapid MPC Using Neural Networks: A Primal-Dual Policy Learning Framework , 2019, IEEE Transactions on Control Systems Technology.

[8]  Mohammad Razeghi-Jahromi,et al.  A stable analytical solution method for car-like robot trajectory tracking and optimization , 2020, IEEE/CAA Journal of Automatica Sinica.

[9]  Anca D. Dragan,et al.  SQIL: Imitation Learning via Reinforcement Learning with Sparse Rewards , 2019, ICLR.

[10]  Ashish Tiwari,et al.  Sherlock - A tool for verification of neural network feedback systems: demo abstract , 2019, HSCC.

[11]  Anonymous Author.  Boosting Structured Prediction for Imitation Learning , 2006 .

[12]  Dimos V. Dimarogonas,et al.  Control Barrier Functions for Multi-Agent Systems Under Conflicting Local Signal Temporal Logic Tasks , 2019, IEEE Control Systems Letters.

[13]  Sarma Vrudhula,et al.  Enabling Incremental Knowledge Transfer for Object Detection at the Edge , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[14]  Ashish Tiwari,et al.  Learning and Verification of Feedback Control Systems using Feedforward Neural Networks , 2018, ADHS.

[15]  Calin Belta,et al.  Sampling-based Motion Planning via Control Barrier Functions , 2019, Proceedings of the 2019 3rd International Conference on Automation, Control and Robots.

[16]  Anca D. Dragan,et al.  SQIL: Imitation Learning via Regularized Behavioral Cloning , 2019, ArXiv.

[17]  Franco Blanchini,et al.  Set invariance in control , 1999, Autom..

[18]  Koushil Sreenath,et al.  Exponential Control Barrier Functions for enforcing high relative-degree safety-critical constraints , 2016, 2016 American Control Conference (ACC).

[19]  Paulo Tabuada,et al.  Control barrier function based quadratic programs with application to adaptive cruise control , 2014, 53rd IEEE Conference on Decision and Control.

[20]  Magnus Egerstedt,et al.  Nonsmooth Barrier Functions With Applications to Multi-Robot Systems , 2017, IEEE Control Systems Letters.

[21]  Martin T. Hagan,et al.  An introduction to the use of neural networks in control systems , 2002 .

[22]  Yan Liu,et al.  Applications of Neural Networks in High Assurance Systems , 2010, Applications of Neural Networks in High Assurance Systems.

[23]  Jyotirmoy V. Deshmukh,et al.  Reasoning about Safety of Learning-Enabled Components in Autonomous Cyber-physical Systems , 2018 .

[24]  Sriram Sankaranarayanan,et al.  Trajectory Tracking Control for Robotic Vehicles Using Counterexample Guided Training of Neural Networks , 2019, ICAPS.

[25]  Stefano Ermon,et al.  Multi-Agent Generative Adversarial Imitation Learning , 2018, NeurIPS.

[26]  Calin Belta,et al.  Control Barrier Functions for Systems with High Relative Degree , 2019, 2019 IEEE 58th Conference on Decision and Control (CDC).

[27]  Paulo Tabuada,et al.  Control Barrier Functions: Theory and Applications , 2019, 2019 18th European Control Conference (ECC).

[28]  SHAKIBA YAGHOUBI,et al.  Worst-case Satisfaction of STL Specifications Using Feedforward Neural Network Controllers: A Lagrange Multipliers Approach , 2019, 2020 Information Theory and Applications Workshop (ITA).

[29]  David M. Bradley,et al.  Boosting Structured Prediction for Imitation Learning , 2006, NIPS.

[30]  Pieter Abbeel,et al.  Apprenticeship learning via inverse reinforcement learning , 2004, ICML.