Control Barriers in Bayesian Learning of System Dynamics

This paper focuses on learning a model of system dynamics online while satisfying safety constraints. Our objective is to avoid offline system identification or hand-specified models and allow a system to safely and autonomously estimate and adapt its own model during operation. Given streaming observations of the system state, we use Bayesian learning to obtain a distribution over the system dynamics. Specifically, we propose a new matrix variate Gaussian process (MVGP) regression approach with an efficient covariance factorization to learn the drift and input gain terms of a nonlinear controlaffine system. The MVGP distribution is then used to optimize the system behavior and ensure safety with high probability, by specifying control Lyapunov function (CLF) and control barrier function (CBF) chance constraints. We show that a safe control policy can be synthesized for systems with arbitrary relative degree and probabilistic CLF-CBF constraints by solving a second order cone program (SOCP). Finally, we extend our design to a self-triggering formulation, adaptively determining the time at which a new control input needs to be applied in order to guarantee safety.

[1]  Nikolai Matni,et al.  On the Sample Complexity of the Linear Quadratic Regulator , 2017, Foundations of Computational Mathematics.

[2]  Naira Hovakimyan,et al.  Safe Feedback Motion Planning: A Contraction Theory and ℒ1-Adaptive Control Based Approach , 2020, 2020 59th IEEE Conference on Decision and Control (CDC).

[3]  Andreas Krause,et al.  Safe Model-based Reinforcement Learning with Stability Guarantees , 2017, NIPS.

[4]  Dorsa Sadigh,et al.  Efficient and Safe Exploration in Deterministic Markov Decision Processes with Unknown Transition Models , 2019, 2019 American Control Conference (ACC).

[5]  D. Sengupta.  Linear models , 2003 .

[6]  Soon-Jo Chung,et al.  Chance-Constrained Trajectory Optimization for Safe Exploration and Learning of Nonlinear Systems , 2020, IEEE Robotics Autom. Lett..

[7]  Andrew Clark,et al.  Control Barrier Functions for Complete and Incomplete Information Stochastic Systems , 2019, 2019 American Control Conference (ACC).

[8]  George J. Pappas,et al.  A Framework for Worst-Case and Stochastic Safety Verification Using Barrier Certificates , 2007, IEEE Transactions on Automatic Control.

[9]  Soon-Jo Chung,et al.  Robust Regression for Safe Exploration in Control , 2019, L4DC.

[10]  Aaron D. Ames,et al.  Adaptive Safety with Control Barrier Functions , 2019, 2020 American Control Conference (ACC).

[11]  S. Sastry.  Nonlinear Systems: Analysis, Stability, and Control , 1999 .

[12]  Jonathan P. How,et al.  Bayesian Nonparametric Adaptive Control Using Gaussian Processes , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[13]  Georgios Fainekos,et al.  Risk-Bounded Control Using Stochastic Barrier Functions , 2021, IEEE Control Systems Letters.

[14]  Marco Pavone,et al.  Evaluating Trajectory Collision Probability through Adaptive Importance Sampling for Safe Motion Planning , 2016, Robotics: Science and Systems.

[15]  Calin Belta,et al.  Control Barrier Functions for Systems with High Relative Degree , 2019, 2019 IEEE 58th Conference on Decision and Control (CDC).

[16]  S. Ghosal,et al.  Posterior consistency of Gaussian process prior for nonparametric binary regression , 2006, math/0702686.

[17]  Aaron D. Ames,et al.  Multi-Rate Control Design Leveraging Control Barrier Functions and Model Predictive Control Policies , 2021, IEEE Control Systems Letters.

[18]  Paulo Tabuada,et al.  Control Barrier Functions: Theory and Applications , 2019, 2019 18th European Control Conference (ECC).

[19]  Tara Javidi,et al.  Gaussian Process bandits with adaptive discretization , 2017, ArXiv.

[20]  Ali-akbar Agha-mohammadi,et al.  Deep Learning Tubes for Tube MPC , 2020, Robotics: Science and Systems.

[21]  Neil D. Lawrence,et al.  Kernels for Vector-Valued Functions: a Review , 2011, Found. Trends Mach. Learn..

[22]  Andrew Gordon Wilson,et al.  GPyTorch: Blackbox Matrix-Matrix Gaussian Process Inference with GPU Acceleration , 2018, NeurIPS.

[23]  Stephen P. Boyd,et al.  Future directions in control in an information-rich world , 2003 .

[24]  Shoya Higa,et al.  MAARS: Machine learning-based Analytics for Automated Rover Systems , 2020, 2020 IEEE Aerospace Conference.

[25]  Andreas Krause,et al.  Safe controller optimization for quadrotors with Gaussian processes , 2015, 2016 IEEE International Conference on Robotics and Automation (ICRA).

[26]  Guang Yang,et al.  Self-triggered Control for Safety Critical Systems Using Control Barrier Functions , 2019, 2019 American Control Conference (ACC).

[27]  Susmit Jha,et al.  Learning Certified Control using Contraction Metric , 2020, CoRL.

[28]  Frank Allgöwer,et al.  CONSTRUCTIVE SAFETY USING CONTROL BARRIER FUNCTIONS , 2007 .

[29]  Koushil Sreenath,et al.  Pointwise Feasibility of Gaussian Process-based Safety-Critical Control under Model Uncertainty , 2021, 2021 60th IEEE Conference on Decision and Control (CDC).

[30]  Koushil Sreenath,et al.  Exponential Control Barrier Functions for enforcing high relative-degree safety-critical constraints , 2016, 2016 American Control Conference (ACC).

[31]  John Lygeros,et al.  Data-Enabled Predictive Control: In the Shallows of the DeePC , 2018, 2019 18th European Control Conference (ECC).

[32]  Soon-Jo Chung,et al.  Robust Controller Design for Stochastic Nonlinear Systems via Convex Optimization , 2020, IEEE Transactions on Automatic Control.

[33]  Bahare Kiumarsi-Khomartash,et al.  Safe Off-policy Reinforcement Learning Using Barrier Functions , 2020, 2020 American Control Conference (ACC).

[34]  Jean-Jacques E. Slotine,et al.  Robust Adaptive Control Barrier Functions: An Adaptive and Data-Driven Approach to Safety , 2021, IEEE Control Systems Letters.

[35]  Paulo Tabuada,et al.  An introduction to event-triggered and self-triggered control , 2012, 2012 IEEE 51st IEEE Conference on Decision and Control (CDC).

[36]  Zoubin Ghahramani,et al.  Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning , 2015, ICML.

[37]  Samuel Coogan,et al.  Synthesis of Control Barrier Functions Using a Supervised Machine Learning Approach , 2020, 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[38]  Dimos V. Dimarogonas,et al.  Control Barrier Functions for Multi-Agent Systems Under Conflicting Local Signal Temporal Logic Tasks , 2019, IEEE Control Systems Letters.

[39]  Panganamala Ramana Kumar,et al.  Cyber–Physical Systems: A Perspective at the Centennial , 2012, Proceedings of the IEEE.

[40]  Yisong Yue,et al.  Learning for Safety-Critical Control with Control Barrier Functions , 2019, L4DC.

[41]  David M. Bradley,et al.  Learning for Autonomous Navigation , 2010, IEEE Robotics & Automation Magazine.

[42]  Lawrence Carin,et al.  Learning Structured Weight Uncertainty in Bayesian Neural Networks , 2017, AISTATS.

[43]  Aaron D. Ames,et al.  Control barrier function based quadratic programs with application to bipedal robotic walking , 2015, 2015 American Control Conference (ACC).

[44]  Koushil Sreenath,et al.  Optimal robust control for constrained nonlinear hybrid systems with application to bipedal locomotion , 2016, 2016 American Control Conference (ACC).

[45]  Samuel Coogan,et al.  A Barrier Function Approach to Finite-Time Stochastic System Verification and Control , 2019, Autom..

[46]  Lisa Turner,et al.  Applications of Second Order Cone Programming , 2012 .

[47]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[48]  Toshimitsu Ushio,et al.  Learning self-triggered controllers with Gaussian processes , 2020, IEEE transactions on cybernetics.

[49]  D. Dimarogonas,et al.  Provably Safe Control of Lagrangian Systems in Obstacle-Scattered Environments , 2020, 2020 59th IEEE Conference on Decision and Control (CDC).

[50]  Gábor Orosz,et al.  End-to-End Safe Reinforcement Learning through Barrier Functions for Safety-Critical Continuous Control Tasks , 2019, AAAI.

[51]  Max Welling,et al.  Structured and Efficient Variational Deep Learning with Matrix Gaussian Posteriors , 2016, ICML.

[52]  Kim Peter Wabersich,et al.  Safe exploration of nonlinear dynamical systems: A predictive safety filter for reinforcement learning , 2018, ArXiv.

[53]  George J. Pappas,et al.  Control Barrier Functions for Unknown Nonlinear Systems using Gaussian Processes* , 2020, 2020 59th IEEE Conference on Decision and Control (CDC).

[54]  Andreas Krause,et al.  Information-Theoretic Regret Bounds for Gaussian Process Optimization in the Bandit Setting , 2009, IEEE Transactions on Information Theory.

[55]  Osbert Bastani,et al.  Safe Reinforcement Learning with Nonlinear Dynamics via Model Predictive Shielding , 2021, 2021 American Control Conference (ACC).

[56]  Calin Belta,et al.  Sampling-based Motion Planning via Control Barrier Functions , 2019, Proceedings of the 2019 3rd International Conference on Automation, Control and Robots.

[57]  Vijay Kumar,et al.  Approximating Explicit Model Predictive Control Using Constrained Neural Networks , 2018, 2018 Annual American Control Conference (ACC).

[58]  Sandra Hirche,et al.  Uniform Error Bounds for Gaussian Process Regression with Application to Safe Control , 2019, NeurIPS.

[59]  David D. Fan,et al.  Bayesian Learning-Based Adaptive Control for Safety Critical Systems , 2019, 2020 IEEE International Conference on Robotics and Automation (ICRA).

[60]  Pieter Abbeel,et al.  BADGR: An Autonomous Self-Supervised Learning-Based Navigation System , 2020, ArXiv.

[61]  Sebastian Junges,et al.  Shielded Decision-Making in MDPs , 2018, ArXiv.

[62]  S. Levine,et al.  Safety Augmented Value Estimation From Demonstrations (SAVED): Safe Deep Model-Based RL for Sparse Cost Robotic Tasks , 2019, IEEE Robotics and Automation Letters.

[63]  Ashwin P. Dani,et al.  Active Sampling based Safe Identification of Dynamical Systems using Extreme Learning Machines and Barrier Certificates , 2019, 2019 International Conference on Robotics and Automation (ICRA).

[64]  Byron Boots,et al.  Agile Autonomous Driving using End-to-End Deep Imitation Learning , 2017, Robotics: Science and Systems.

[65]  Alexander Rakhlin,et al.  Near optimal finite time identification of arbitrary linear dynamical systems , 2018, ICML.

[66]  Jean-Jacques E. Slotine,et al.  Adaptive Nonlinear Control With Contraction Metrics , 2021, IEEE Control Systems Letters.

[67]  Dimos V. Dimarogonas,et al.  Learning Control Barrier Functions from Expert Demonstrations , 2020, 2020 59th IEEE Conference on Decision and Control (CDC).

[68]  Marco Pavone,et al.  Robust online motion planning via contraction theory and convex optimization , 2017, 2017 IEEE International Conference on Robotics and Automation (ICRA).

[69]  Li Wang,et al.  Safe Learning of Quadrotor Dynamics Using Barrier Certificates , 2017, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[70]  Sham M. Kakade,et al.  Plan Online, Learn Offline: Efficient Learning and Exploration via Model-Based Control , 2018, ICLR.

[71]  Evangelos A. Theodorou,et al.  Safe Optimal Control Using Stochastic Barrier Functions and Deep Forward-Backward SDEs , 2020, ArXiv.

[72]  Carl E. Rasmussen,et al.  PILCO: A Model-Based and Data-Efficient Approach to Policy Search , 2011, ICML.

[73]  Sriram Sankaranarayanan,et al.  Training Neural Network Controllers Using Control Barrier Functions in the Presence of Disturbances , 2020, 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC).

[74]  Calin Belta,et al.  Adaptive Control Barrier Functions for Safety-Critical Systems , 2020, ArXiv.

[75]  Mrdjan Jankovic,et al.  Robust control barrier functions for constrained stabilization of nonlinear systems , 2018, Autom..

[76]  Byron Boots,et al.  Learning a Contact-Adaptive Controller for Robust, Efficient Legged Locomotion , 2020, Conference on Robot Learning.

[77]  Marco Pavone,et al.  Meta-Learning Priors for Efficient Online Bayesian Regression , 2018, WAFR.

[78]  Thomas Lew,et al.  Safe Model-Based Meta-Reinforcement Learning: A Sequential Exploration-Exploitation Framework , 2020, ArXiv.

[79]  Sebastian Trimpe,et al.  Event-triggered Learning , 2019, Autom..

[80]  Sandra Hirche,et al.  Posterior Variance Analysis of Gaussian Processes with Application to Average Learning Curves , 2019, ArXiv.

[81]  Masahiro Ono,et al.  Convex Optimization over Sequential Linear Feedback Policies with Continuous-time Chance Constraints , 2019, 2019 IEEE 58th Conference on Decision and Control (CDC).

[82]  Massimo Franceschetti,et al.  Learning-based attacks in cyber-physical systems. , 2018 .

[83]  Sandra Hirche,et al.  Feedback Linearization Based on Gaussian Processes With Event-Triggered Online Learning , 2019, IEEE Transactions on Automatic Control.

[84]  Torsten Koller,et al.  Learning-based Model Predictive Control for Safe Exploration and Reinforcement Learning , 2019, ArXiv.

[85]  Aaron D. Ames,et al.  Formal Test Synthesis for Safety-Critical Autonomous Systems based on Control Barrier Functions , 2020, 2020 59th IEEE Conference on Decision and Control (CDC).

[86]  Dimitra Panagou,et al.  Control-Lyapunov and Control-Barrier Functions based Quadratic Program for Spatio-temporal Specifications , 2019, 2019 IEEE 58th Conference on Decision and Control (CDC).

[87]  Paulo Tabuada,et al.  Robustness of Control Barrier Functions for Safety Critical Control , 2016, ADHS.

[88]  Sergey Levine,et al.  End-to-End Training of Deep Visuomotor Policies , 2015, J. Mach. Learn. Res..

[89]  Carl E. Rasmussen,et al.  Gaussian processes for machine learning , 2005, Adaptive computation and machine learning.

[90]  R. Adler.  The Geometry of Random Fields , 2009 .

[91]  Jaime F. Fisac,et al.  A General Safety Framework for Learning-Based Control in Uncertain Robotic Systems , 2017, IEEE Transactions on Automatic Control.

[92]  Riccardo Bonalli,et al.  Chance-Constrained Sequential Convex Programming for Robust Trajectory Optimization , 2020, 2020 European Control Conference (ECC).

[93]  Debasish Ghose,et al.  High-Relative Degree Stochastic Control Lyapunov and Barrier Functions , 2020, ArXiv.

[94]  Paulo Tabuada,et al.  Control Barrier Function Based Quadratic Programs for Safety Critical Systems , 2016, IEEE Transactions on Automatic Control.

[95]  Paulo Tabuada,et al.  Realizing simultaneous lane keeping and adaptive speed regulation on accessible mobile robot testbeds , 2017, 2017 IEEE Conference on Control Technology and Applications (CCTA).

[96]  Massimo Franceschetti,et al.  Probabilistic Safety Constraints for Learned High Relative Degree System Dynamics , 2020, L4DC.