Linear Model Predictive Safety Certification for Learning-Based Control

While it has been repeatedly shown that learning-based controllers can provide superior performance, they often lack of safety guarantees. This paper aims at addressing this problem by introducing a model predictive safety certification (MPSC) scheme for linear systems with additive disturbances. The scheme verifies safety of a proposed learning-based input and modifies it as little as necessary in order to keep the system within a given set of constraints. Safety is thereby related to the existence of a model predictive controller (MPC) providing a feasible trajectory towards a safe target set. A robust MPC formulation accounts for the fact that the model is generally uncertain in the context of learning, which allows for proving constraint satisfaction at all times under the proposed MPSC strategy. The MPSC scheme can be used in order to expand any potentially conservative set of safe states and we provide an iterative technique for enlarging the safe set. Finally, a practical data-based design procedure for MPSC is proposed using scenario optimization.

[1]  Marco C. Campi,et al.  The Exact Feasibility of Randomized Solutions of Uncertain Convex Programs , 2008, SIAM J. Optim..

[2]  Stephen P. Boyd,et al.  Fast Model Predictive Control Using Online Optimization , 2010, IEEE Transactions on Control Systems Technology.

[3]  S. Shankar Sastry,et al.  Provably safe and robust learning-based model predictive control , 2011, Autom..

[4]  D. Mayne,et al.  On the Minimal Robust Positively Invariant Set for Linear Difference Inclusions , 2005, Proceedings of the 44th IEEE Conference on Decision and Control.

[5]  Franco Blanchini,et al.  Set invariance in control , 1999, Autom..

[6]  Christopher K. I. Williams,et al.  Gaussian Processes for Machine Learning (Adaptive Computation and Machine Learning) , 2005 .

[7]  Frank Allgöwer,et al.  A quasi-infinite horizon nonlinear model predictive control scheme with guaranteed stability , 1997, 1997 European Control Conference (ECC).

[8]  Kim Peter Wabersich,et al.  Scalable synthesis of safety certificates from data with application to learning-based control , 2018, 2018 European Control Conference (ECC).

[9]  Torsten Koller,et al.  Learning-based Model Predictive Control for Safe Exploration and Reinforcement Learning , 2019, ArXiv.

[10]  Francesco Borrelli,et al.  Learning Model Predictive Control for Iterative Tasks: A Computationally Efficient Approach for Linear System , 2017, ArXiv.

[11]  Shane Legg,et al.  Human-level control through deep reinforcement learning , 2015, Nature.

[12]  Benjamin Recht,et al.  Simple random search provides a competitive approach to reinforcement learning , 2018, ArXiv.

[13]  Javier García,et al.  A comprehensive survey on safe reinforcement learning , 2015, J. Mach. Learn. Res..

[14]  Basil Kouvaritakis,et al.  Robust Tube MPC for Linear Systems With Multiplicative Uncertainty , 2015, IEEE Transactions on Automatic Control.

[15]  Jaime F. Fisac,et al.  A General Safety Framework for Learning-Based Control in Uncertain Robotic Systems , 2017, IEEE Transactions on Automatic Control.

[16]  Yuval Tassa,et al.  Learning human behaviors from motion capture by adversarial imitation , 2017, ArXiv.

[17]  Jaime F. Fisac,et al.  Reachability-based safe learning with Gaussian processes , 2014, 53rd IEEE Conference on Decision and Control.

[18]  David Q. Mayne,et al.  Invariant approximations of the minimal robust positively Invariant set , 2005, IEEE Transactions on Automatic Control.

[19]  Andrea Carron,et al.  Safe Learning for Distributed Systems with Bounded Uncertainties , 2017 .

[20]  H. ChenT,et al.  A Quasi-Infinite Horizon Nonlinear Model Predictive Control Scheme with Guaranteed Stability * , 1998 .

[21]  David Bremner,et al.  Primal—Dual Methods for Vertex and Facet Enumeration , 1998, Discret. Comput. Geom..

[22]  Andrew Y. Ng,et al.  Policy search via the signed derivative , 2009, Robotics: Science and Systems.

[23]  Biao Huang,et al.  System Identification , 2000, Control Theory for Physicists.

[24]  J. Geromel,et al.  A new discrete-time robust stability condition , 1999 .

[25]  Giuseppe Carlo Calafiore,et al.  The scenario approach to robust control design , 2006, IEEE Transactions on Automatic Control.

[26]  Manfred Morari,et al.  Efficient interior point methods for multistage problems arising in receding horizon control , 2012, 2012 IEEE 51st IEEE Conference on Decision and Control (CDC).

[27]  Petre Stoica,et al.  Decentralized Control , 2018, The Control Systems Handbook.