Model-free Learning for Safety-critical Control Systems: A Reference Governor Approach

This paper describes a learning-based approach to operating safety-critical control systems. A reference governor is an add-on scheme used to guard the nominal system against violation of pre-specified constraints by modifying set-point commands. A learning algorithm is developed in this paper to evolve the reference governor parametrization to gradually improve its performance in terms of response speed. In particular, the learning algorithm does not rely on an explicit model of the control system, i.e., it is model-free, and guarantees constraint satisfaction for all time, both during and after learning. The approach is applied to a case study of ground vehicle rollover avoidance to illustrate its functionality and characteristics.

[1]  Nan Li,et al.  Model-free Learning to Avoid Constraint Violations: An Explicit Reference Governor Approach , 2019, 2019 American Control Conference (ACC).

[2]  Stefano Di Cairano,et al.  Reference and command governors for systems with constraints: A survey on theory and applications , 2017, Autom..

[3]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[4]  Torsten Koller,et al.  Learning-based Model Predictive Control for Safe Exploration and Reinforcement Learning , 2019, ArXiv.

[5]  Ilya Kolmanovsky,et al.  Reference Governor Strategies for Vehicle Rollover Avoidance , 2016, IEEE Transactions on Control Systems Technology.

[6]  Angela P. Schoellig,et al.  Robust Constrained Learning-based NMPC enabling reliable mobile robot path tracking , 2016, Int. J. Robotics Res..

[7]  Gábor Orosz,et al.  End-to-End Safe Reinforcement Learning through Barrier Functions for Safety-Critical Continuous Control Tasks , 2019, AAAI.

[8]  Andreas Krause,et al.  Safe Model-based Reinforcement Learning with Stability Guarantees , 2017, NIPS.

[9]  J. Kocijan,et al.  Gaussian process model based predictive control , 2004, Proceedings of the 2004 American Control Conference.

[10]  Nan Li,et al.  A Reference Governor for Nonlinear Systems With Disturbance Inputs Based on Logarithmic Norms and Quadratic Programming , 2020, IEEE Transactions on Automatic Control.

[11]  S. Shankar Sastry,et al.  Provably safe and robust learning-based model predictive control , 2011, Autom..

[12]  Javier García,et al.  A comprehensive survey on safe reinforcement learning , 2015, J. Mach. Learn. Res..